基于熵值过滤和类质心优化的无监督域适应

doi:10.11896/jsjkx.230500144

Abstract

Abstract: As one of the emerging research directions in the field of machine learning,unsupervised domainadaptation mainly uses source domain supervision information to assist the learning of unlabeled target domains.Recently,many unsupervised domain adaptation methods have been proposed,but there are still some deficiencies in relation mining.Specifically,existing methods usually adopt a consistent processing strategy for target domain samples,while ignoring the discrepancy in target domain samples in relation mining.Therefore,this paper proposes a novel method called entropy filtering and class centroid optimization(EFCO).The proposed method utilizes a generative adversarial network architecture to label target domain samples.With the obtained pseudo-labels,the sample entropy value is calculated and compared with a predefined threshold to further categorize target domain samples.Simple samples are assigned pseudo-labels,while difficult samples are classified using the idea of contrastive learning.By combining source domain data and simple samples,a more robust classifier is learned to classify difficult samples,and class centroids of the source and target domains are obtained.Inter-domain and intra-domain discrepancies are minimized by optimizing inter-domain contrastive alignment and instance contrastive alignment.Finally,it is compared with several advanced domain adaptation methods on three standard data sets,and the results indicate that the performance of the proposed method outperforms the comparison methods.

Key words: Transfer learning, Unsupervised domain adaptation, Adversarial learning, Contrastive learning, Class centroid optimization

CLC Number:

TP181

TIAN Qing, LU Zhanghu, YANG Hong. Unsupervised Domain Adaptation Based on Entropy Filtering and Class Centroid Optimization[J].Computer Science, 2024, 51(7): 345-353.

References

[1]SUN B,SAENKO K.Deep coral:Correlation alignment for deep domain adaptation[C]//Proceedings of European Conference on Computer Vision.Springer,2016:443-450.
[2]TIAN Q,SUN H,MA C,et al.Heterogeneous Domain Adaptation With Structure and Classification Space Alignment[J].IEEE Transactions on Cybernetics,2022,52(10):10328-10338.
[3]LONG M,CAO Z,WANG J,et al.Conditional adversarial do-main adaptation[J].Advances in Neural Information Processing Systems,2018,31:1647-1657.
[4]PEI Z,CAO Z,LONG M,et al.Multi-Adversarial domain adaptation[C]//Proceedings of AAAI Conference on Artificial Intelligence.AAAI Press,2018:3934-3941.
[5]GANIN Y,LEMPITSKY V.Unsupervised domain adaptationby backpropagation[C]//Proceedings of International Confe-rence on Machine Learning.PMLR,2015:1180-1189.
[6]TIAN Q,ZHU Y,SUN H,CHEN S,et al.Unsupervised Domain Adaptation Through Dynamically Aligning Both the Feature and Label Spaces[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,32(12):8562-8573.
[7]LONG M,ZHU H,WANG J,et al.Unsupervised domain adaptation with residual transfer networks[J].Advances in Neural Information Processing Systems,2016,29:136-144.
[8]TIAN Q,PENG S,MA T.Source-free Unsupervised DomainAdaptation with Trusted Pseudo Samples[J].ACM Transactions on Intelligent Systems and Technology,2023,14(2):2157-6904.
[9]OORD A,LI Y,VINYALS O.Representation learning with contrastive predictive coding[J].arXiv:1807.03748,2018.
[10]HE K,FAN H,WU Y,et al.Momentum contrast for unsupervised visual representation learning[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2020:9726-9735.
[11]KANG G,JIANG L,YANG Y,et al.Contrastive adaptation network for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2019:4893-4902.
[12]CHEN T,KORNBLITH S,NOROUZI M,et al.A simpleframework for contrastive learning of visual representations[C]//Proceedings of International Conference on Machine Learning.PMLR,2020:1597-1607.
[13]GRILL J B,STRUB F,ALTCHE F,et al.Bootstrap your own latent-a new approach to self-supervised learning[J].Advances in Neural Information Processing Systems,2020,33:21271-21284.
[14]SINGH A,CHAKRABORTY O,VARSHNEY A,et al.Semi-supervised action recognition with temporal contrastive learning[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2021:10389-10399.
[15]XIE S,ZHENG Z,CHEN L,et al.Learning semantic representations for unsupervised domain adaptation[C]//Proceedings of International Conference on Machine Learning.PMLR,2018:5423-5432.
[16]PINHEIRO P O.Unsupervised domain adaptation with similarity learning[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:8004-8013.
[17]PAN Y,YAO T,LI Y,et al.Transferrable prototypical net-works for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2019:2239-2247.
[18]YAN H,LI Z,WANG Q,et al.Weighted and class-specific ma-ximum mean discrepancy for unsupervised domain adaptation[J].IEEE Transactions on Multimedia,2019,22(9):2420-2433.
[19]HU L,KAN M,SHAN S,et al.Unsupervised domain adaptation with hierarchical gradient synchronization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2020:4043-4052.
[20]LU Y,ZHU Q,ZHANG B,et al.Weighted correlation embedding learning for domain adaptation[J].IEEE Transactions on Image Processing,2022,31:5303-5316.
[21]CHEN C,XIE W,HUANG W,et al.Progressive feature alignment for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2019:627-636.
[22]WANG X,LI L,YE W,et al.Transferable attention for domain adaptation[C]//Proceedings of AAAI Conference on Artificial Intelligence.AAAI Press,2019:5345-5352.
[26]ZHANG W,OUYANG W,LI W,et al.Collaborative and adversarial network for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2018:3801-3809.
[24]ZHU Y,ZHUANG F,WANG J,et al.Deep subdomain adapta-tion network for image classification[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(4):1713-1722.
[25]DENG Z,LUO Y,ZHU J.Cluster alignment with a teacher for unsupervised domain adaptation[C]//Proceedings of IEEE International Conference on Computer Vision.IEEE,2019:9944-9953.
[26]WANG Q,BRECKON T.Unsupervised domain adaptation via structured prediction based selective pseudo-labeling[C]//Proceedings of AAAI Conference on Artificial Intelligence.AAAI Press,2020:6243-6250.
[27]WANG M,DENG W.Cycle label-consistent networks for unsupervised domain adaptation[J].Neurocomputing,2021,422:186-199.
[28]MORERIO P,VOLPI R,RAGONESI R,et al.Generative pseudo-label refinement for unsupervised domain adaptation[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision.IEEE,2020:3130-3139.
[29]LITRICO M,DEL BUE A,MORERIO P.Guiding Pseudo-La-bels With Uncertainty Estimation for Source-Free Unsupervised Domain Adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2023:7640-7650.
[30]CHU T,LIU Y,DENG J,et al.Denoised Maximum Classifier Discrepancy for Source-Free Unsupervised Domain Adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.AAAI Press,2022:472-480.
[31]TIAN Y,KRISHNAN D,ISOLA P.Contrastive multiview co-ding[C]//Proceedings of European Conference on Computer Vision.Springer,2020:776-794.
[32]KHOSLA P,TETERWAK P,WANG C,et al.Supervised contrastive learning[J].Advances in Neural Information Processing Systems,2020,33:18661-18673.
[33]GE Y X,ZHU F,CHEN D,et al.Self-paced contrastive learning with hybrid memory for domain adaptive object re-id[J].Advances in Neural Information Processing Systems,2020,33:11309-11321.
[34]THOTA M,LEONTIDIS G.Contrastive domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.2021:2209-2218.
[35]LIU W,FERSTL D,SCHULTER S,et al.Domain adaptationfor semantic segmentation via patch-wise contrastive learning[J].arXiv:2104.11056,2021.
[36]GRETTON A,BORGWARDT K M,RASCH M J,et al.A kernel two-sample test[J].Journal of Machine Learning Research,2012,13(1):723-773.
[37]HUANG J,GUAN D,XIAO A,et al.Category contrast for un-supervised domain adaptation in visual tasks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2022:1203-1214.
[38]PAN S,YANG Q.A survey on transfer learning[J].IEEETransactions on Knowledge and Data Engineering,2010,22(10):1345-1359.
[39]HOFFMAN J,TZENG E,PARK T,et al.Cycada:Cycle-con-sistent adversarial domain adaptation[C]//Proceedings of International Conference on Machine Learning.PMLR,2018:1989-1998.
[40]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].IEEE,1998,86(11):2278-2324.
[41]HULL J.A database for handwritten text recognition research[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1994,16(5):550-554.
[42]NETZER Y,WANG T,COATES A,et al.Reading digits in na-tural images with unsupervised feature learning[J].NIPS Workshop on Deep Learning & Unsupervised Feature Learning.2011.
[43]SAENLO K,KULIS B,FRITZ M,et al.Adapting visual category models to new domains[C]//Proceedings of European Conference on Computer Vision.Springer,2010:213-226.
[44]VENKATESWARA H,EUSBIO J,CHAKRABORTY S,et al.Deep hashing network for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2017:5018-5027.
[45]GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adversarial training of neural networks[J].TheJournal of Machine Learning Research,2016,17(1):2096-2030.
[46]TZENG E,HOFFMAN J,SAENKO K,et al.Adversarial dis-criminative domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2017:7167-7176.
[47]LONG M,CAO Y,WANG J,et al.Learning transferable fea-tures with deep adaptation networks[C]//Proceedings of International Conference on Machine Learning.PMLR,2015:97-105.
[48]SAITO K,WATANABE K,USHIKU Y,et al.Maximum classifier discrepancy for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2018:3723-3732.
[49]XIAO N,ZHANG L.Dynamic weighted learning for unsuper-vised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2021:15242-15251.
[50]LIU H,LONG M,WANG J,et al.Transferable adversarialtraining:A general approach to adapting deep classifiers[C]//Proceedings of International Conference on Machine Learning.PMLR,2019:4013-4022.
[51]LONG M,ZHU H,WANG J,et al.Deep transfer learning with joint adaptation networks[C]//Proceedings of International Conference on Machine Learning.PMLR,2017:2208-2217.
[52]LI S,SONG S,WU C.Layer-wise domain correction for unsupervised domain adaptation[J].Frontiers of Information Technology Electronic Engineering,2018,19(1):91-103.
[53]LIU M,TUZEL O.Coupled generative adversarial networks[J].Advances in Neural Information Processing Systems,2016,29:469-477.
[54]XU R,LI G,YANG J,et al.Larger norm more transferable:An adaptive feature norm approach for unsupervised domain adaptation[C]//Proceedings of IEEE International Conference on Computer Vision.IEEE,2019:1426-1435.
[55]LI M,ZHAI Y,LUO Y,et al.Enhanced transport distance for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2020:13936-13944.
[56]DU Z,LI J,SU H,et al.Cross-domain gradient discrepancy mi-nimization for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2021:3937-3946.
[57]WANG H,TIAN J,LI S,et al.Structure-conditioned adversarial learning for unsupervised domain adaptation[J].Neurocompu-ting,2022,497:216-226.
[58]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer So-ciety,2016:770-778.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Unsupervised Domain Adaptation Based on Entropy Filtering and Class Centroid Optimization

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	HU Haibo, YANG Dan, NIE Tiezheng, KOU Yue. Graph Contrastive Learning Incorporating Multi-influence and Preference for Social Recommendation [J]. Computer Science, 2024, 51(7): 146-155.
[2]	CAO Yan, ZHU Zhenfeng. DRSTN:Deep Residual Soft Thresholding Network [J]. Computer Science, 2024, 51(6A): 230400112-7.
[3]	ZHANG Xinrui, YANG Jian, WANG Zhan. Thai Speech Synthesis Based on Cross-language Transfer Learning and Joint Training [J]. Computer Science, 2024, 51(6A): 230500174-7.
[4]	YU Bihui, TAN Shuyue, WEI Jingxuan, SUN Linzhuang, BU Liping, ZHAO Yiman. Vision-enhanced Multimodal Named Entity Recognition Based on Contrastive Learning [J]. Computer Science, 2024, 51(6): 198-205.
[5]	LI Yilin, SUN Chengsheng, LUO Lin, JU Shenggen. Aspect-based Sentiment Classification for Word Information Enhancement Based on Sentence Information [J]. Computer Science, 2024, 51(6): 299-308.
[6]	WANG Jiahao, FU Yifu, FENG Hainan, REN Yuheng. Indoor Location Algorithm in Dynamic Environment Based on Transfer Learning [J]. Computer Science, 2024, 51(5): 277-283.
[7]	CHEN Runhuan, DAI Hua, ZHENG Guineng, LI Hui , YANG Geng. Urban Electricity Load Forecasting Method Based on Discrepancy Compensation and Short-termSampling Contrastive Loss [J]. Computer Science, 2024, 51(4): 158-164.
[8]	LIAO Jinzhi, ZHAO Hewei, LIAN Xiaotong, JI Wenliang, SHI Haiming, ZHAO Xiang. Contrastive Graph Learning for Cross-document Misinformation Detection [J]. Computer Science, 2024, 51(3): 14-19.
[9]	HUANG Kun, SUN Weiwei. Traffic Speed Forecasting Algorithm Based on Missing Data [J]. Computer Science, 2024, 51(3): 72-80.
[10]	YANG Bo, LUO Jiachen, SONG Yantao, WU Hongtao, PENG Furong. Time Series Clustering Method Based on Contrastive Learning [J]. Computer Science, 2024, 51(2): 63-72.
[11]	XU Jie, WANG Lisong. Contrastive Clustering with Consistent Structural Relations [J]. Computer Science, 2023, 50(9): 123-129.
[12]	HU Shen, QIAN Yuhua, WANG Jieting, LI Feijiang, LYU Wei. Super Multi-class Deep Image Clustering Model Based on Contrastive Learning [J]. Computer Science, 2023, 50(9): 192-201.
[13]	LI Xiang, FAN Zhiguang, LIN Nan, CAO Yangjie, LI Xuexiang. Self-supervised Learning for 3D Real-scenes Question Answering [J]. Computer Science, 2023, 50(9): 220-226.
[14]	YANG Lin, YANG Jian, CAI Haoran, LIU Cong. Vietnamese Speech Synthesis Based on Transfer Learning [J]. Computer Science, 2023, 50(8): 118-124.
[15]	WANG Mingxia, XIONG Yun. Disease Diagnosis Prediction Algorithm Based on Contrastive Learning [J]. Computer Science, 2023, 50(7): 46-52.