Computer Science ›› 2025, Vol. 52 ›› Issue (9): 212-219.doi: 10.11896/jsjkx.240700159

• Database & Big Data & Data Science • Previous Articles     Next Articles

Source-free Domain Adaptation Method Based on Pseudo Label Uncertainty Estimation

HUANG Chao, CHENG Chunling, WANG Youkang   

  1. School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China
  • Received:2024-07-24 Revised:2024-10-22 Online:2025-09-15 Published:2025-09-11
  • About author:HUANG Chao,born in 1999,postgra-duate.His main research interests include deep learning and domain adaptation.
    CHENG Chunling,born in 1972,professor.Her main research interests include data mining and data management.
  • Supported by:
    National Natural Science Foundation of China(61972201).

Abstract: Most of the source-free domain adaptive methods are self-supervised learning based on pseudo-labels to solve the source-free limitation.However,these methods overlook the impact of the clustering structure of target sample feature distributions and the uncertainty of samples near the decision boundary on pseudo-label noise during the generation process,which reduces the model’s performance.Therefore,this paper proposes a source-free domain adaptive method based on pseudo-label uncertainty estimation.Firstly,multiple perturbations are introduced to the model’s feature extractor parameters to simulate variations in source knowledge resulting from data fine-tuning.The similarity of target sample feature distributions under different perturbed models is utilized to assess the generalization uncertainty of the source knowledge.Moreover,an extreme value entropy is proposed to quantify the latent information uncertainty within the target domain,where distinct entropy calculation methods are applied based on the difference between the highest and second-highest prediction probabilities.Secondly,the target samples are divided into reliable samples and unreliable samples according to two kinds of uncertainty.For reliable samples,self-supervised learning is employed,using their prediction probabilities as weights to update sample features into class prototypes.Furthermore,historical class prototypes are incorporated to improve the stability of the class prototypes.Contrast learning is applied to unreliable samples to bring them closer to similar class prototypes.Compared with several baseline models,the classification accuracy is improved on three public benchmark datasets:Office-31,Office-Home and VisDA-C,and the effectiveness of the method is verified.

Key words: Domain adaptive, Source-free data, Uncertainty estimation, Information entropy, Contrastive Learning

CLC Number: 

  • TP391.1
[1]ZHANG L,FU J,WANG S,et al.Guide Subspace Learning for Unsupervised Domain Adaptation[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(9):3374-3388.
[2]TIAN J,ZHANG J,WEN L,et al.VDM-DA:Virtual DomainModeling for Source Data-Free Domain Adaptation[J].IEEE Transactions on Circuits and Systems for Video Technology,2021,32(6):3749-3760.
[3]LI R,JIAO Q,CAO W,et al.Model Adaptation:Unsupervised Domain Adaptation without Source Data[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Computer Society,2020:9641-9650.
[4]LIANG J,HU D,FENG J.Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation[C]//Proceedings of International Conference on Machine Learning.New York:ACM,2020:6028-6039.
[5]YANG S,WANG Y,WEIJER J,et al.Exploiting the Intrinsic Neighborhood Structure for Source-Free Domain Adaptation[J].Advances in Neural Information Processing Systems,2021,34:29393-29405.
[6]LITRICO M,BUE A,MORERIO P.Guiding Pseudo-labels with Uncertainty Estimation for Source-free Unsupervised Domain Adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Computer Society,2023:7640-7650.
[7]WU J,WANG G,GU R,et al.UPL-SFDA:Uncertainty-aware Pseudo Label Guided Source-Free DomainAdaptation for Medical Image Segmentation[J].IEEE Transactions on Medical Imaging,2023,42(12):3932-3943.
[8]CHEN X,YANG H,ZHANG H,et al.Uncertainty-AwarePseudo-Label Filtering for Source-Free Unsupervised Domain Adaptation[J].Neurocomputing,2024,575:127190.
[9]LI J,YU Z,DU Z,et al.A Comprehensive Survey on Source-free Domain Adaptation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2024,46(8):5743-5762.
[10]CUI C,ZHANG C,LIU Z,et al.Adversarial Source Generation for Source-Free Domain Adaptation[J].IEEE Transactions on Circuits and Systems for Video Technology,2024,34(6):4887-4898.
[11]HE J,WU L,TAO C,et al.Source-Free Domain Adaptationwith Unrestricted Source Hypothesis[J].Pattern Recognition,2024,149:110246.
[12]XIA H,ZHAO H,DING Z.Adaptive Adversarial Network for Source-free Domain Adaptation[C]//Proceedings of IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE Computer Society,2021:9010-9019.
[13]TIAN Q,PENG S,MA T.Source-Free Unsupervised Domain Adaptation with Trusted Pseudo Samples[J].ACM Transactions on Intelligent Systems and Technology,2023,14(2):1-17.
[14]DING Y,SHENG L,LIANG J,et al.ProxyMix:Proxy-based Mixup Training with Label Refinery for Source-Free Domain Adaptation[J].Neural Networks,2023,167:92-103.
[15]KIM Y,CHO D,HAN K,et al.Domain Adaptation withoutSource Data[J].IEEE Transactions on Artificial Intelligence,2021,2(6):508-518.
[16]AHMED W,MORERIO P,MURINO V.Cleaning Noisy Labels by Negative Ensemble Learning for Source-Free Unsupervised Domain Adaptation[C]//Proceedings of IEEE Winter Confe-rence on Applications of Computer Vision.Piscataway,NJ:IEEE Computer Society,2022:1616-1625.
[17]KIM Y,YIM J,YUN J,et al.NLNL:Negative Learning forNoisy Labels[C]//Proceedings of IEEE International Confe-rence on Computer Vision.Piscataway,NJ:IEEE Computer So-ciety,2019:101-110.
[18]ZHANG C,BENGIO S,HARDT M,et al.Understanding Deep Learning (still) Requires Rethinking Generalization[J].Communications of the ACM,2021,64(3):107-115.
[19]MA X,HUANG H,WANG Y,et al.Normalized Loss Functions for Deep Learning with Noisy Labels[C]//Proceedings of International Conference on Machine Learning.New York:ACM,2020:6543-6553.
[20]WEI H,FENG L,CHEN X,et al.Combating Noisy Labels by Agreement:A Joint Training Method with Co-Regularization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Computer Society,2020:13726-13735.
[21]SAENKO K,KULIS B,FRITZ M,et al.Adapting Visual Category Models to New Domains[C]//Proceedings of European Conference on Computer Vision.Berlin:Springer,2010:213-226.
[22]VENKATESWARA H,EUSEBIO J,CHAKRABORTY S,et al.Deep Hashing Network for Unsupervised Domain Adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Computer Society,2017:5018-5027.
[23]PENG X,USMAN B,KAUSHIK N,et al.VisDA:The Visual Domain Adaptation Challenge[J].arXiv:1710.06924,2017.
[24]GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-Adversarial Training of Neural Networks[J].Journal of Machine Learning Research,2016,17(59):1-35.
[25]XU R,LI G,YANG J,et al.Larger norm more transferable:An adaptive feature norm approach for unsupervised domain adaptation[C]//Proceedings of IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE Computer Society,2019:1426-1435.
[26]LIANG J,HU D,FENG J.Domain Adaptation with Auxiliary Target Domain-Oriented Classifier[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Computer Society,2021:16632-16642.
[27]HE K,ZHANG X,REN S,et al.Deep Residual Learning for Image Recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Computer Society,2016:770-778.
[28]TANG S,SU W,YE M,et al.Source-Free Domain Adaptation with Frozen Multimodal Foundation Model[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Computer Society,2024:23711-23720.
[1] ZHANG Shiju, GUO Chaoyang, WU Chengliang, WU Lingjun, YANG Fengyu. Text Clustering Approach Based on Key Semantic Driven and Contrastive Learning [J]. Computer Science, 2025, 52(8): 171-179.
[2] ZHANG Taotao, XIE Jun, QIAO Pingjuan. Specific Emitter Identification Based on Progressive Self-training Open Set Domain Adaptation [J]. Computer Science, 2025, 52(7): 279-286.
[3] YE Jiale, PU Yuanyuan, ZHAO Zhengpeng, FENG Jue, ZHOU Lianmin, GU Jinjing. Multi-view CLIP and Hybrid Contrastive Learning for Multimodal Image-Text Sentiment Analysis [J]. Computer Science, 2025, 52(6A): 240700060-7.
[4] FU Shufan, WANG Zhongqing, JIANG Xiaotong. Zero-shot Stance Detection in Chinese by Fusion of Emotion Lexicon and Graph ContrastiveLearning [J]. Computer Science, 2025, 52(6A): 240500051-7.
[5] LI Jianghui, DING Haiyan, LI Weihua. Prediction of Influenza A Antigenicity Based on Few-shot Contrastive Learning [J]. Computer Science, 2025, 52(6A): 240800053-6.
[6] LIU Yuanhong, WU Yubin. Local Linear Embedding Algorithm Based on Probability Model and Information Entropy [J]. Computer Science, 2025, 52(6A): 240500021-8.
[7] LIU Yufei, XIAO Yanhui, TIAN Huawei. PRNU Fingerprint Purification Algorithm for Open Environment [J]. Computer Science, 2025, 52(6): 187-199.
[8] CHEN Yadang, GAO Yuxuan, LU Chuhan, CHE Xun. Saliency Mask Mixup for Few-shot Image Classification [J]. Computer Science, 2025, 52(6): 256-263.
[9] WU Pengyuan, FANG Wei. Study on Graph Collaborative Filtering Model Based on FeatureNet Contrastive Learning [J]. Computer Science, 2025, 52(5): 139-148.
[10] MIAO Zhuang, CUI Haoran, ZHANG Qiyang, WANG Jiabao, LI Yang. Restoration of Atmospheric Turbulence-degraded Images Based on Contrastive Learning [J]. Computer Science, 2025, 52(5): 171-178.
[11] TIAN Qing, KANG Lulu, ZHOU Liangyu. Class-incremental Source-free Domain Adaptation Based on Multi-prototype Replay andAlignment [J]. Computer Science, 2025, 52(3): 206-213.
[12] YUAN Ye, CHEN Ming, WU Anbiao, WANG Yishu. Graph Anomaly Detection Model Based on Personalized PageRank and Contrastive Learning [J]. Computer Science, 2025, 52(2): 80-90.
[13] XU Jiucheng, ZHANG Shan, BAI Qing, MA Miaoxian. Attribute Reduction Algorithm Based on Fuzzy Neighborhood Relative Decision Entropy [J]. Computer Science, 2025, 52(2): 165-172.
[14] LIU Yanlun, XIAO Zheng, NIE Zhenyu, LE Yuquan, LI Kenli. Case Element Association with Evidence Extraction for Adjudication Assistance [J]. Computer Science, 2025, 52(2): 222-230.
[15] YE Lishuo, HE Zhixue. Multi-granularity Time Series Contrastive Learning Method Incorporating Time-Frequency Features [J]. Computer Science, 2025, 52(1): 170-182.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!