Computer Science ›› 2019, Vol. 46 ›› Issue (3): 221-226.doi: 10.11896/j.issn.1002-137X.2019.03.033

• Artificial Intelligence • Previous Articles     Next Articles

Clustering Assist Feature Alignment for Unsupervised Domain Adaptation

YUAN Ding, WANG Qian, DENG Li-wei   

  1. (College of Computer Science,Chongqing University,Chongqing 400044,China)
  • Received:2018-10-10 Revised:2019-01-21 Online:2019-03-15 Published:2019-03-22

Abstract: Supervised deep learning can reach good results in the areas with large amounts of labeled data,but the rea-lity is that there are only a lot of unlabeled data in many areas.How to take advantages of large amounts of unlabeled data has become a key issue in the development of deep learning.Domain adaptation is an effective way to solve this problem.At present,domain adaptation methods based on adversarial training have achieved a good effect.This method uses domain classification loss to align the feature distribution of source domain,and target domain and reduce the difference of distribution between the feature representations of two domain,so the classifier trained with source domain data can be applied to target domain data.The existing domain adaptation method trains the model on the features after domain adaptation and does not make full use of the original information of the target domain data.When the differences between two domains are large,the intra-domain discriminability of target domain features will be reduced.In view of the disadvantages of the present methods,this paper proposed a method for clustering target domain data to assist feature alignment(CAFA-DA) based on the adversarial discriminative domain adaptation (ADDA).Pseudo-labels of target domain data are obtained by clustering and the feature encoder training is constrained in the domain adaptation stage,and the original information of the target domain data is used to improve the discriminability of target domain features.Classifiers trained in the two processes of clustering and domain adaptation are used for ensemble learning and high confidence samples are trained to improve the final effect of the model.The CAFA-DA can be applied to any domain adaption method based on adversarial loss.Finally,this paper compared CAFA-DA with several advanced domain adaption methods on four standard domain adaption data sets.The results show that the accuracy of the CAFA-DA method is better than other methods.Compared with the ADDA method,the results of two comparative experiments are improved by 3.2% and 17.2% respectively.

Key words: Clustering, Domain adaption, Ensemble learning, Feature alignment, Feature representation

CLC Number: 

  • TP181
[1]BEN-DAVID S,BLITZER J,CRAMMER K,et al.A theory of learning from different domains.Machine learning,2010,79(1-2):151-175.
[2]TZENG E,HOFFMAN J,SAENKO K,et al.Adversarial dis-
criminative domain adaptation[C]∥Computer Vision and Pattern Recognition (CVPR).IEEE,2017:4.
[3]TZENG E,HOFFMAN J,ZHANG N,et al.Deep Domain Confusion:Maximizing for Domain Invariance.arXiv:1412.3474,2014.
[4]LONG M,CAO Y,WANG J,et al.Learning transferable fea-
tures with deep adaptation networks.arXiv:1502.02791,2015.
[5]LONG M,ZHU H,WANG J,et al.Deep transfer learning with joint adaptation networks[C]∥Proceedings of the 34th International Conference on Machine Learning-Volume 70.2017:2208-2217.
[6]GRETTON A,BORGWARDT K M,RASCH M J,et al.A kernel two-sample test.Journal of Machine Learning Research,2012,13(1):723-773.
[7]GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adver-
sarial training of neural networks.The Journal of Machine Learning Research,2016,17(1):2096-2030.
[8]BOUSMALIS K,TRIGEORGIS G,SILBERMAN N,et al.Domain separation networks[C]∥Advances in Neural Information Processing Systems.IEEE,2016:343-351.
[9]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Gene-
rative adversarial nets[C]∥Advances in Neural Information Processing Systems.IEEE,2014:2672-2680.
[10]GULRAJANI I,AHMED F,ARJOVSKY M,et al.Improved
training of wasserstein gans[C]∥Advances in Neural Information Processing Systems.IEEE,2017:5767-5777.
[11]SHU R,BUI H H,NARUI H,et al.A DIRT-T Approach to Unsupervised Domain Adaptation.arXiv:1802.08735,2018.
[12]TAKERU M,SHIN-ICHI M,SHIN I,et al.Virtual Adversarial Training:A Regularization Method for Supervised and Semi-Supervised Learning.IEEE Transactions on Pattern Analysis and Machine Intelligence,2018:1.
[13]SAITO K,USHIKU Y,HARADA T.Asymmetric tri-training for unsupervised domain adaptation.arXiv:1702.08400,2017.
[14]CHEN C,XIE W,XU T,et al.Progressive Feature Alignment for Unsupervised Domain Adaptation.arXiv:1811.08585,2018.
[15]KUMAR A,SATTIGERI P,WADHAWAN K,et al.Co-regularized Alignment for Unsupervised Domain Adaptation[C]∥Advances in Neural Information Processing Systems.IEEE,2018:9367-9378.
[16]GRANDVALET Y,BENGIO Y.Semi-supervised learning by entropy minimization[C]∥Advances in Neural Information Processing Systems.IEEE,2005:529-536.
[17]SHIMODAIRA H.Improving predictive inference under cova-
riate shift by weighting the log-likelihood function.Journal of statistical planning and inference,2000,90(2):227-244.
[18]MANSOUR Y,MOHRI M,ROSTAMIZADEH A.Domain ada-
ptation:Learning bounds and algorithms.arXiv:0902.3430,2009.
[19]CHEN X,DUAN Y,HOUTHOOFT R,et al.Infogan:Interpretable representation learning by information maximizingge-nerative adversarial nets[C]∥Advances in Neural Information Processing Systems.IEEE,2016:2172-2180.
[20]TZENG E,HOFFMAN J,DARRELL T,et al.Simultaneous
deep transfer across domains and tasks[C]∥Proceedings of the IEEE International Conference on Computer Vision.IEEE,2015:4068-4076.
[21]GANIN Y,LEMPITSKY V.Unsupervised domain adaptation by backpropagation.arXiv:1409.7495,2014.
[22]GHIFARY M,KLEIJN W B,ZHANG M,et al.Deep reconstruction-classification networks for unsupervised domain adaptation[C]∥European Conference on Computer Vision.Cham:Springer,2016:597-613.
[23]LIU M Y,TUZEL O.Coupled generative adversarial networks[C]∥Advances in Neural Information Processing Systems.IEEE,2016:469-477.
[24]BOUSMALIS K,SILBERMAN N,DOHAN D,et al.Unsupervised pixel-level domain adaptation with generative adversarial networks[C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017:7.
[25]TAIGMAN Y,POLYAK A,WOLF L.Unsupervised cross-do-
main image generation.arXiv:1611.02200,2016.
[26]CHAPELLE O,ZIEN A.Semi-Supervised Classification by Low Density Separation[C]∥AISTATS.2005:57-64.
[27]DAI Z,YANG Z,YANG F,et al.Good semi-supervised learning that requires a bad gan[C]∥Advances in Neural Information Processing Systems.IEEE,2017:6510-6520.
[28]ZHOU Z H,LI M.Tri-training:Exploiting unlabeled data using three classifiers.IEEE Transactions on knowledge and Data Engineering,2005,17(11):1529-1541.
[29]CARON M,BOJANOWSKI P,JOULIN A,et al.Deep Clustering for Unsupervised Learning of Visual Features//Computer Vision-ECCV 2018.Cham:Springer,2018:139-156.
[30]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet Classification with Deep Convolutional Neural Networks[C]∥Advances in Neural Information Processing Systems.IEEE,2012:1097-1105.
[31]SRIVASTAVA N,HINTON G,KRIZHEVSKY A,et al.Dropout:a simple way to prevent neural networks from overfitting.The Journal of Machine Learning Research,2014,15(1):1929-1958.
[32]JOHNSON J,DOUZE M,JGOU H.Billion-scale similarity
search with gpus.arXiv:1702.08734,2017.
[33]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based
learning applied to document recognition.Proceedings of the IEEE,1998,86(11):2278-2324.
[34]ARBELAEZ P,MAIRE M,FOWLKES C,et al.Contour detection and hierarchical image segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(5):898-916.
[35]NETZER Y,WANG T,COATES A,et al.Reading digits in na-
tural images with unsupervised feature learning[C]∥NIPS Workshop on Deep Learning and Unsupervised Feature Lear-ning.2011:5.
[36]WANG Y,WANG Q,LV Q F,et al.An Improved K-means Algorithm Based on Initial Clustering Center Optimization and Weighted Between Dimension.Journal of Chongqing University of Technology(Natural Science),2013,27(4):77-80.(in Chinese)
王越,王泉,吕奇峰,等. 基于初始聚类中心优化和维间加权的改进K-means算法. 重庆理工大学学报(自然科学),2013,27(4):77-80.
[1] CHAI Hui-min, ZHANG Yong, FANG Min. Aerial Target Grouping Method Based on Feature Similarity Clustering [J]. Computer Science, 2022, 49(9): 70-75.
[2] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients [J]. Computer Science, 2022, 49(9): 183-193.
[3] YU Shu-hao, ZHOU Hui, YE Chun-yang, WANG Tai-zheng. SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion [J]. Computer Science, 2022, 49(6A): 256-260.
[4] MAO Sen-lin, XIA Zhen, GENG Xin-yu, CHEN Jian-hui, JIANG Hong-xia. FCM Algorithm Based on Density Sensitive Distance and Fuzzy Partition [J]. Computer Science, 2022, 49(6A): 285-290.
[5] CHEN Jing-nian. Acceleration of SVM for Multi-class Classification [J]. Computer Science, 2022, 49(6A): 297-300.
[6] LIN Xi, CHEN Zi-zhuo, WANG Zhong-qing. Aspect-level Sentiment Classification Based on Imbalanced Data and Ensemble Learning [J]. Computer Science, 2022, 49(6A): 144-149.
[7] KANG Yan, WU Zhi-wei, KOU Yong-qi, ZHANG Lan, XIE Si-yu, LI Hao. Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution [J]. Computer Science, 2022, 49(6A): 150-158.
[8] Ran WANG, Jiang-tian NIE, Yang ZHANG, Kun ZHU. Clustering-based Demand Response for Intelligent Energy Management in 6G-enabled Smart Grids [J]. Computer Science, 2022, 49(6): 44-54.
[9] WANG Yu-fei, CHEN Wen. Tri-training Algorithm Based on DECORATE Ensemble Learning and Credibility Assessment [J]. Computer Science, 2022, 49(6): 127-133.
[10] CHEN Jia-zhou, ZHAO Yi-bo, XU Yang-hui, MA Ji, JIN Ling-feng, QIN Xu-jia. Small Object Detection in 3D Urban Scenes [J]. Computer Science, 2022, 49(6): 238-244.
[11] HAN Hong-qi, RAN Ya-xin, ZHANG Yun-liang, GUI Jie, GAO Xiong, YI Meng-lin. Study on Cross-media Information Retrieval Based on Common Subspace Classification Learning [J]. Computer Science, 2022, 49(5): 33-42.
[12] XING Yun-bing, LONG Guang-yu, HU Chun-yu, HU Li-sha. Human Activity Recognition Method Based on Class Increment SVM [J]. Computer Science, 2022, 49(5): 78-83.
[13] ZHU Zhe-qing, GENG Hai-jun, QIAN Yu-hua. Line-Segment Clustering Algorithm for Chemical Structure [J]. Computer Science, 2022, 49(5): 113-119.
[14] ZHANG Yu-jiao, HUANG Rui, ZHANG Fu-quan, SUI Dong, ZHANG Hu. Study on Affinity Propagation Clustering Algorithm Based on Bacterial Flora Optimization [J]. Computer Science, 2022, 49(5): 165-169.
[15] ZUO Yuan-lin, GONG Yue-jiao, CHEN Wei-neng. Budget-aware Influence Maximization in Social Networks [J]. Computer Science, 2022, 49(4): 100-109.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!