Computer Science ›› 2025, Vol. 52 ›› Issue (2): 116-124.doi: 10.11896/jsjkx.240600004

• Database & Big Data & Data Science • Previous Articles     Next Articles

Multi-source-free Domain Adaptation Based on Source Model Contribution Quantization

TIAN Qing1,2,3, LIU Xiang1, WANG Bin1, YU Jiangsen1, SHEN Jiashuo1   

  1. 1 School of Software,Nanjing University of Information Science and Technology,Nanjing 210044,China
    2 Wuxi Institute of Technology,Nanjing University of Information Science and Technology,Wuxi,Jiangsu 214000,China
    3 State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China
  • Received:2024-06-02 Revised:2024-09-07 Online:2025-02-15 Published:2025-02-17
  • About author:TIAN Qing,born in 1984,Ph.D,professor,is a senior member of CCF(No.33364S).His main research interests include machine learning,pattern recognition and computer vision.
  • Supported by:
    National Natural Science Foundation of China(62176128),Natural Science Foundation of Jiangsu Province(BK20231143),Open Projects Program of State Key Laboratory for Novel Software Technology of Nanjing University(KFKT2022B06),Fundamental Research Funds for the Central Universities(NJ2022028)and Qing Lan Project of Jiangsu Province.

Abstract: As a new research direction in the field of machine learning,multi-source-free domain adaptation aims to transfer knowledge from multiple source domain models to the target domain,so as to achieve accurate prediction of target domain samples.Essentially,the key to solving multi-source-free domain adaptation lies in how to quantify the contribution of multiple source models to the target domain and utilize the diverse knowledge in the source models to adapt to the target domain.To address these issues,this paper proposes a multi-source-free domain adaptation method based on source model contribution quantization(SMCQ).Specifically,a source model transferability perception is proposed to quantify the transferability contribution of the source model,enabling the effective allocation of adaptive weights for target domain models.Additionally,an information maximization method is introduced to reduce cross-domain distributional discrepancies and mitigate model degradation.Subsequently,a credible partition global alignment approach is proposed to divide high-confidence and low-confidence samples to cope with the noisy environment caused by domain differences,effectively reduce the risk of incorrect label assignments.In addition,a sample local consistency loss is also introduced to mitigate the impact of pseudo-label noise on clustering errors of low-confidence samples.Finally,experiments conducted on multiple datasets validate the effectiveness of the proposed method.

Key words: Multi-source-free domain adaptation, Multi-model contribution quantization, Source-model transferable perception, Information maximum, Credible partition of samples

CLC Number: 

  • TP391
[1]TIAN Q,ZHU Y N,SUN H Y,et al.Unsupervised domain adaptation through dynamically aligning both the feature and label spaces[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,32(12):8562-8573.
[2]TIAN Q,CHU Y,SUN H Y,et al.Survey on partial domain adaptation[J].Journal of Software,2023,34(12):5597-5613.
[3]TIAN Q,SUN H Y,MA C,et al.Heterogeneous domain adaptation with structure and classification space alignment[J].IEEE Transactions on Cybernetics,2021,52(10):10328-10338.
[4]LONG M S,CAO Y,WANG J M,et al.Learning transferablefeatures with deep adaptation networks[C]//International Conference on Machine Learning.PMLR,2015:97-105.
[5]LONG M S,CAO Y,CAO Z J,et al.Transferable representation learning with deep adaptation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,41(12):3071-3085.
[6]PENG X C,BAI Q X,XIA X D,et al.Moment matching formulti-source domain adaptation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:1406-1415.
[7]GANIN Y,LEMPITSKY V.Unsupervised domain adaptationby backpropagation[C]//International Conference on Machine Learning.PMLR,2015:1180-1189.
[8]GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adversarial training of neural networks[J].Journal of Machine Lear-ning Research,2016,17(1):2096-2030.
[9]TZENG E,HOFFMAN J,SAENKO K,et al.Adversarial dis-criminative domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:7167-7176.
[10]KIM Y,CHO D,HAN K,et al.Domain adaptation withoutsource data[J].IEEE Transactions on Artificial Intelligence,2021,2(6):508-518.
[11]LIANG J,HU D P,FENG J S.Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation[C]//International Conference on Machine Learning.PMLR,2020:6028-6039.
[12]LI R,JIAO Q F,CAO W M,et al.Model adaptation:unsupervised domain adaptation without source data[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:9641-9650.
[13]YANG S Q,WANG Y X,VAN DE WEIJER J,et al.Unsupervised domain adaptation without source data by casting a bait[J].arXiv:2010.12427,2020.
[14]LIN H B,ZHANG Y F,QIU Z,et al.Prototype-guided continu-al adaptation for class-incremental unsupervised domain adaptation[C]//European Conference on Computer Vision.Cham:Springer Nature Switzerland,2022:351-368.
[15]KARIM N,MITHUN N C,RAJVANSHI A,et al.C-SFDA:A Curriculum Learning Aided Self-Training Framework for Efficient Source Free Domain Adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:24120-24131.
[16]KIM Y,CHO D,HONG S.Towards privacy-preserving domain adaptation[J].IEEE Signal Processing Letters,2020,27:1675-1679.
[17]WANG F,HAN Z Y,GONG Y S,et al.Exploring domain-invariant parameters for source free domain adaptation[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:7151-7160.
[18]LIANG J,HU D P,FENG J S,et al.Dine:Domain adaptation from single and multiple black-box predictors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:8003-8013.
[19]ZHANG H J,ZHANG Y B,JIA K,et al.Unsupervised domainadaptation of black-box source models[J].arXiv:2101.02839,2021.
[20]KUNDU J N,VENKAT N,BADU R V.Universal source-free domain adaptation[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2020:4544-4553.
[21]WANG F,HAN Z Y,ZHANG Z Y,et al.MHPL:Minimumhappy points learning for active source free domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:20008-20018.
[22]AHMED S M,RAYCHAUDHURI D S,PAUL S,et al.Unsupervised multi-source domain adaptation without access to source data[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10103-10112.
[23]DONG J H,FANG Z,LIU A J,et al.Confident anchor-induced multi-source free domain adaptation[J].Advances in Neural Information Processing Systems,2021,34:2848-2860.
[24]HAN Z Y,ZHANG Z Y,WANG F,et al.Discriminability andtransferability estimation:a bayesian source importance estimation approach for multi-source-free domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2023:7811-7820.
[25]LI J,DU Z K,ZHU L,et al.Divergence-agnostic unsupervised domain adaptation by adversarial attacks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(11):8196-8211.
[26]YE M,ZHANG J,OUYANG J,et al.Source data-free unsuper-vised domain adaptation for semantic segmentation[C]//Proceedings of the 29th ACM International Conference on Multimedia.2021:2233-2242.
[27]DING N,XU Y,TANG Y,et al.Source-free domain adaptation via distribution estimation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:7212-7222.
[28]TIAN Q,MA C,ZHANG F Y,et al.Source-free unsupervised domain adaptation with sample transport learning[J].Journal of Computer Science and Technology,2021,36(3):606-616.
[29]KURMI V K,SUBRAMANIAN V K,NAMBOODIRI V P.Domain impression:A sourcedata free domain adaptation method[C]//Proceedings of the IEEE/CVF winter Conference on Applications of Computer Vision.2021:615-625.
[30]ZHANG Z,CHEN W,CHENG H,et al.Divide and contrast:Source-free domain adaptation via adaptive contrastive learning[J].Advances in Neural Information Processing Systems,2022,35:5137-5149.
[31]YANG S,VAN DE WEIJER J,HERRANZ L,et al.Exploiting the intrinsic neighborhood structure for source-free domain adaptation[J].Advances in Neural Information Processing Systems,2021,34:29393-29405.
[32]YANG S,JUI S,VAN DE WEIJER J.Attracting and disper-sing:A simple approach for source-free domain adaptation[J].Advances in Neural Information Processing Systems,2022,35:5802-5815.
[33]GUO J,SHAH D J,BARZILAY R.Multi-source domain adaptation with mixture of experts[J].arXiv:1809.02256,2018.
[34]YANG L,BALAJI Y,LIM S N,et al.Curriculum manager for source selection in multi-source domain adaptation[C]//Computer Vision-ECCV 2020:16th European Conference,Glasgow,UK,August 23-28,2020,Proceedings,Part XIV 16.Springer International Publishing,2020:608-624.
[35]ZHAO S,WANG G,ZHANG S,et al.Multi-source distilling domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:12975-12983.
[36]ZHAO H,ZHANG S,WU G,et al.Adversarial multiple source domain adaptation[J].Advances in Neural Information Proces-sing Systems,2018,31:8559-8570.
[37]XU R,CHEN Z,ZUO W,et al.Deep cocktail network:Multi-source unsupervised domain adaptation with category shift[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3964-3973.
[38]LI Y,CARLSON D E.Extracting relationships by multi-domain matching[J].Advances in Neural Information Processing Systems,2018,31:6799-6810.
[39]WANG H,YANG W,LIN Z,et al.TMDA:Task-specific multi-source domain adaptation via clustering embedded adversarial training[C]//2019 IEEE International Conference on Data Mi-ning(ICDM).IEEE,2019:1372-1377.
[40]SHEN M,BU Y,WORNELL G W.On balancing bias and variance in unsupervised multi-source-free domain adaptation[C]//International Conference on Machine Learning.PMLR,2023:30976-30991.
[41]HOFFMAN J,MOHRI M,ZHANG N.Algorithms and theory for multiple-source adaptation[J].Advances in Neural Information Processing Systems,2018,31:8256-8266.
[42]ZHAO S,LI B,YUE X,et al.Multi-source domain adaptation for semantic segmentation[J].Advances in Neural Information Processing Systems,2019,32:7285-7298.
[43]LI K Q Y,LU J,ZUO H,et al.Multi-source contribution lear-ning for domain adaptation[J].IEEE Transactions on Neural Networks and Learning Systems,2021,33(10):5293-5307.
[44]ZHANG J,ZHOU W E,CHEN X Q,et al.Multisource selective transfer framework in multiobjective optimization problems[J].IEEE Transactions on Evolutionary Computation,2019,24(3):424-438.
[45]BRIDLE J,HEADING A,MACKAY D.Unsupervised classi-fiers,mutual information and ‘phantom targets’[J].Advances in Neural Information Processing Systems,1991,4:1096-1101.
[46]KRAUSE A,PERONA P,GOMES R.Discriminative clustering by regularized information maximization[J].Advances in Neural Information Processing Systems,2010,23:775-783.
[47]SAENKO K,KULIS B,FRITZ M,et al.Adapting visual category models to new domains[C]//Computer Vision-ECCV 2010:11th European Conference on Computer Vision,Heraklion,Crete,Greece,September 5-11,2010,Proceedings,Part IV 11.Springer Berlin Heidelberg,2010:213-226.
[48]GONG B Q,SHI Y,SHA F,et al.Geodesic flow kernel for unsupervised domain adaptation[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2012:2066-2073.
[49]VENKATESWARA H,EUSEBIO J,CHAKRABORTY S,et al.Deep hashing network for unsupervised domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:5018-5027.
[50]HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[51]XU R J,CHEN Z L,ZUO W M,et al.Deep cocktail network:Multi-source unsupervised domain adaptation with category shift[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3964-3973.
[52]WANG H,XU M H,NI B,et al.Learning to combine:Know-ledge aggregation for multi-source domain adaptation[C]//Computer Vision-ECCV 2020:16th European Conference,Glasgow,UK,August 23-28,2020,Proceedings,Part VIII 16.Springer International Publishing,2020:727-744.
[53]KIM Y,CHO D,PANDA P,et al.Progressive domain adaptation from a source pre-trained model[J].arXiv:2007.01524,2020.
[1] LI Jiawei , DENG Yuandan, CHEN Bo. Domain UML Model Automatic Construction Based on Fine-tuning Qwen2 [J]. Computer Science, 2025, 52(6A): 240900155-4.
[2] ZOU Rui, YANG Jian, ZHANG Kai. Low-resource Vietnamese Speech Synthesis Based on Phoneme Large Language Model andDiffusion Model [J]. Computer Science, 2025, 52(6A): 240700138-6.
[3] BAI Yuntian, HAO Wenning, JIN Dawei. Study on Open-domain Question Answering Methods Based on Retrieval-augmented Generation [J]. Computer Science, 2025, 52(6A): 240800141-7.
[4] YIN Baosheng, ZONG Chen. Research on Semantic Fusion of Chinese Polysemous Words Based on Large LanguageModel [J]. Computer Science, 2025, 52(6A): 240400139-7.
[5] HU Caishun. Study on Named Entity Recognition Algorithms in Audit Domain Based on Large LanguageModels [J]. Computer Science, 2025, 52(6A): 240700190-4.
[6] WANG Yicheng, NING Tai, LIU Xinyu, LUO Ye. Position-aware Based Multi-modality Lung Cancer Survival Prediction Method [J]. Computer Science, 2025, 52(6A): 240500089-8.
[7] LONG Xiao, HUANG Wei, HU Kai. Bi-MI ViT:Bi-directional Multi-level Interaction Vision Transformer for Lung CT ImageClassification [J]. Computer Science, 2025, 52(6A): 240700183-6.
[8] GUAN Xin, YANG Xueyong, YANG Xiaolin, MENG Xiangfu. Tumor Mutation Prediction Model of Lung Adenocarcinoma Based on Pathological [J]. Computer Science, 2025, 52(6A): 240700010-8.
[9] TAN Jiahui, WEN Chenyan, HUANG Wei, HU Kai. CT Image Segmentation of Intracranial Hemorrhage Based on ESC-TransUNet Network [J]. Computer Science, 2025, 52(6A): 240700030-9.
[10] GUO Xiaoli, LI Qifeng, LIU Yu, ZHANG Jun, ZHAO Hongtao, YANG Gan, JIANG Ruixiang, YU Ligen. Study on Diagnosis Model of Livestock and Poultry Disease Based on Improved TF-IIGM Algorithm [J]. Computer Science, 2025, 52(6A): 240700029-7.
[11] FAN Xing, ZHOU Xiaohang, ZHANG Ning. Review on Methods and Applications of Short Text Similarity Measurement in Social Media Platforms [J]. Computer Science, 2025, 52(6A): 240400206-8.
[12] ZHAO Zheyu, WANG Zhongqing, WANG Hongling. Commodity Attribute Classification Method Based on Dual Pre-training [J]. Computer Science, 2025, 52(6A): 240500127-8.
[13] LIN Nan, LIU Zhihui, YANG Cong. Named Entity Recognition Algorithm Based on Pre-training Model and Bidirectional TwoDimensional Convolution [J]. Computer Science, 2025, 52(6A): 240700143-6.
[14] YE Jiale, PU Yuanyuan, ZHAO Zhengpeng, FENG Jue, ZHOU Lianmin, GU Jinjing. Multi-view CLIP and Hybrid Contrastive Learning for Multimodal Image-Text Sentiment Analysis [J]. Computer Science, 2025, 52(6A): 240700060-7.
[15] WANG Baohui, XU Boren, LI Chang’ao, YE Zihao. Study on Algorithm for Keyword Extraction from WeChat Conversation Text [J]. Computer Science, 2025, 52(6A): 240700105-8.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!