Computer Science ›› 2019, Vol. 46 ›› Issue (3): 221-226.doi: 10.11896/j.issn.1002-137X.2019.03.033

• Artificial Intelligence • Previous Articles     Next Articles

Clustering Assist Feature Alignment for Unsupervised Domain Adaptation

YUAN Ding, WANG Qian, DENG Li-wei   

  1. (College of Computer Science,Chongqing University,Chongqing 400044,China)
  • Received:2018-10-10 Revised:2019-01-21 Online:2019-03-15 Published:2019-03-22

Abstract: Supervised deep learning can reach good results in the areas with large amounts of labeled data,but the rea-lity is that there are only a lot of unlabeled data in many areas.How to take advantages of large amounts of unlabeled data has become a key issue in the development of deep learning.Domain adaptation is an effective way to solve this problem.At present,domain adaptation methods based on adversarial training have achieved a good effect.This method uses domain classification loss to align the feature distribution of source domain,and target domain and reduce the difference of distribution between the feature representations of two domain,so the classifier trained with source domain data can be applied to target domain data.The existing domain adaptation method trains the model on the features after domain adaptation and does not make full use of the original information of the target domain data.When the differences between two domains are large,the intra-domain discriminability of target domain features will be reduced.In view of the disadvantages of the present methods,this paper proposed a method for clustering target domain data to assist feature alignment(CAFA-DA) based on the adversarial discriminative domain adaptation (ADDA).Pseudo-labels of target domain data are obtained by clustering and the feature encoder training is constrained in the domain adaptation stage,and the original information of the target domain data is used to improve the discriminability of target domain features.Classifiers trained in the two processes of clustering and domain adaptation are used for ensemble learning and high confidence samples are trained to improve the final effect of the model.The CAFA-DA can be applied to any domain adaption method based on adversarial loss.Finally,this paper compared CAFA-DA with several advanced domain adaption methods on four standard domain adaption data sets.The results show that the accuracy of the CAFA-DA method is better than other methods.Compared with the ADDA method,the results of two comparative experiments are improved by 3.2% and 17.2% respectively.

Key words: Domain adaption, Feature alignment, Clustering, Ensemble learning, Feature representation

CLC Number: 

  • TP181
[1] BEN-DAVID S,BLITZER J,CRAMMER K,et al.A theory of learning from different domains.Machine learning,2010,79(1-2):151-175.
[2] TZENG E,HOFFMAN J,SAENKO K,et al.Adversarial dis-criminative domain adaptation[C]∥Computer Vision and Pattern Recognition (CVPR).IEEE,2017:4.
[3] TZENG E,HOFFMAN J,ZHANG N,et al.Deep Domain Confusion:Maximizing for Domain Invariance.arXiv:1412.3474,2014.
[4] LONG M,CAO Y,WANG J,et al.Learning transferable fea-tures with deep adaptation networks.arXiv:1502.02791,2015.
[5] LONG M,ZHU H,WANG J,et al.Deep transfer learning with joint adaptation networks[C]∥Proceedings of the 34th International Conference on Machine Learning-Volume 70.2017:2208-2217.
[6] GRETTON A,BORGWARDT K M,RASCH M J,et al.A kernel two-sample test.Journal of Machine Learning Research,2012,13(1):723-773.
[7] GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adver-sarial training of neural networks.The Journal of Machine Learning Research,2016,17(1):2096-2030.
[8] BOUSMALIS K,TRIGEORGIS G,SILBERMAN N,et al.Domain separation networks[C]∥Advances in Neural Information Processing Systems.IEEE,2016:343-351.
[9] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Gene-rative adversarial nets[C]∥Advances in Neural Information Processing Systems.IEEE,2014:2672-2680.
[10] GULRAJANI I,AHMED F,ARJOVSKY M,et al.Improvedtraining of wasserstein gans[C]∥Advances in Neural Information Processing Systems.IEEE,2017:5767-5777.
[11] SHU R,BUI H H,NARUI H,et al.A DIRT-T Approach to Unsupervised Domain Adaptation.arXiv:1802.08735,2018.
[12] TAKERU M,SHIN-ICHI M,SHIN I,et al.Virtual Adversarial Training:A Regularization Method for Supervised and Semi-Supervised Learning.IEEE Transactions on Pattern Analysis and Machine Intelligence,2018:1.
[13] SAITO K,USHIKU Y,HARADA T.Asymmetric tri-training for unsupervised domain adaptation.arXiv:1702.08400,2017.
[14] CHEN C,XIE W,XU T,et al.Progressive Feature Alignment for Unsupervised Domain Adaptation.arXiv:1811.08585,2018.
[15] KUMAR A,SATTIGERI P,WADHAWAN K,et al.Co-regularized Alignment for Unsupervised Domain Adaptation[C]∥Advances in Neural Information Processing Systems.IEEE,2018:9367-9378.
[16] GRANDVALET Y,BENGIO Y.Semi-supervised learning by entropy minimization[C]∥Advances in Neural Information Processing Systems.IEEE,2005:529-536.
[17] SHIMODAIRA H.Improving predictive inference under cova-riate shift by weighting the log-likelihood function.Journal of statistical planning and inference,2000,90(2):227-244.
[18] MANSOUR Y,MOHRI M,ROSTAMIZADEH A.Domain ada-ptation:Learning bounds and algorithms.arXiv:0902.3430,2009.
[19] CHEN X,DUAN Y,HOUTHOOFT R,et al.Infogan:Interpretable representation learning by information maximizingge-nerative adversarial nets[C]∥Advances in Neural Information Processing Systems.IEEE,2016:2172-2180.
[20] TZENG E,HOFFMAN J,DARRELL T,et al.Simultaneousdeep transfer across domains and tasks[C]∥Proceedings of the IEEE International Conference on Computer Vision.IEEE,2015:4068-4076.
[21] GANIN Y,LEMPITSKY V.Unsupervised domain adaptation by backpropagation.arXiv:1409.7495,2014.
[22] GHIFARY M,KLEIJN W B,ZHANG M,et al.Deep reconstruction-classification networks for unsupervised domain adaptation[C]∥European Conference on Computer Vision.Cham:Springer,2016:597-613.
[23] LIU M Y,TUZEL O.Coupled generative adversarial networks[C]∥Advances in Neural Information Processing Systems.IEEE,2016:469-477.
[24] BOUSMALIS K,SILBERMAN N,DOHAN D,et al.Unsupervised pixel-level domain adaptation with generative adversarial networks[C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017:7.
[25] TAIGMAN Y,POLYAK A,WOLF L.Unsupervised cross-do-main image generation.arXiv:1611.02200,2016.
[26] CHAPELLE O,ZIEN A.Semi-Supervised Classification by Low Density Separation[C]∥AISTATS.2005:57-64.
[27] DAI Z,YANG Z,YANG F,et al.Good semi-supervised learning that requires a bad gan[C]∥Advances in Neural Information Processing Systems.IEEE,2017:6510-6520.
[28] ZHOU Z H,LI M.Tri-training:Exploiting unlabeled data using three classifiers.IEEE Transactions on knowledge and Data Engineering,2005,17(11):1529-1541.
[29] CARON M,BOJANOWSKI P,JOULIN A,et al.Deep Clustering for Unsupervised Learning of Visual Features//Computer Vision-ECCV 2018.Cham:Springer,2018:139-156.
[30] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet Classification with Deep Convolutional Neural Networks[C]∥Advances in Neural Information Processing Systems.IEEE,2012:1097-1105.
[31] SRIVASTAVA N,HINTON G,KRIZHEVSKY A,et al.Dropout:a simple way to prevent neural networks from overfitting.The Journal of Machine Learning Research,2014,15(1):1929-1958.
[32] JOHNSON J,DOUZE M,JGOU H.Billion-scale similaritysearch with gpus.arXiv:1702.08734,2017.
[33] LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition.Proceedings of the IEEE,1998,86(11):2278-2324.
[34] ARBELAEZ P,MAIRE M,FOWLKES C,et al.Contour detection and hierarchical image segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(5):898-916.
[35] NETZER Y,WANG T,COATES A,et al.Reading digits in na-tural images with unsupervised feature learning[C]∥NIPS Workshop on Deep Learning and Unsupervised Feature Lear-ning.2011:5.
[36] WANG Y,WANG Q,LV Q F,et al.An Improved K-means Algorithm Based on Initial Clustering Center Optimization and Weighted Between Dimension.Journal of Chongqing University of Technology(Natural Science),2013,27(4):77-80.(in Chinese)王越,王泉,吕奇峰,等. 基于初始聚类中心优化和维间加权的改进K-means算法. 重庆理工大学学报(自然科学),2013,27(4):77-80.
[1] QIN Yue, DING Shi-fei. Survey of Semi-supervised Clustering [J]. Computer Science, 2019, 46(9): 15-21.
[2] SUN Guo-dao, ZHOU Zhi-xiu, LI Si, LIU Yi-peng, LIANG Rong-hua. Spatio-Temporal Evolution of Geographical Topics [J]. Computer Science, 2019, 46(8): 42-49.
[3] ZHANG Hui-bing, ZHONG Hao, HU Xiao-li. User Reviews Clustering Method Based on Topic Analysis [J]. Computer Science, 2019, 46(8): 50-55.
[4] YANG Zhen, WANG Hong-jun. Important Location Identification of Mobile Users Based on Trajectory Division and Density Clustering Method [J]. Computer Science, 2019, 46(8): 23-27.
[5] CAI Li, LI Ying-zi, JIANG Fang, LIANG Yu. Study on Clustering Mining of Imbalanced Data Fusion Towards Urban Hotspots [J]. Computer Science, 2019, 46(8): 16-22.
[6] WANG Jing, QIU Xiao-he. Advanced MDS-MAP Localization Algorithm with Clustering and Fusion Compensation Strategy [J]. Computer Science, 2019, 46(8): 145-151.
[7] SUN Shu-ya, FANG Huan, FANG Xian-wen. Log-induced Morphological Fragments Process Clustering Method [J]. Computer Science, 2019, 46(8): 71-77.
[8] LUO Xi, FAN Jiu-lun, YU Hai-yan, LIANG Dan. Selection of Cutset Threshold for Cutset-type Possibilistic C-means Clustering Based on Shadowed Set [J]. Computer Science, 2019, 46(8): 249-254.
[9] ANG Li-fang, SHI Chao-yu, LIN Su-zhen, QIN Pin-le, GAO Yuan. Multi-modal Medical Image Fusion Based on Joint Patch Clustering of Adaptive Dictionary Learning [J]. Computer Science, 2019, 46(7): 238-245.
[10] LI Xiao-guang, SHAO Chao. Density Peak Clustering Algorithm Based on Grid Data Center [J]. Computer Science, 2019, 46(6A): 457-460.
[11] WANG Nan, SUN Shan-wu. UAV Fault Recognition Based on Semi-supervised Clustering [J]. Computer Science, 2019, 46(6A): 192-195.
[12] HOU Yuan-yuan, HE Ru-han, LI Min, CHEN Jia. Clothing Image Retrieval Method Combining Convolutional Neural Network Multi-layerFeature Fusion and K-Means Clustering [J]. Computer Science, 2019, 46(6A): 215-221.
[13] LIANG Ping-yuan, LI Jie, PENG Jiao, WANG Hui. Research on 3D Dynamic Clustering Routing Algorithm Based on Cooperative MIMO for UWSN [J]. Computer Science, 2019, 46(6A): 336-342.
[14] LI Jian-jun, HOU Yue, YANG Yu. User Interest Recommendation Model Based on Context Awareness [J]. Computer Science, 2019, 46(6A): 502-506.
[15] HUANG Hai-yan, LIU Xiao-ming, SUN Hua-yong, YANG Zhi-cai. Application of Clustering Analysis Algorithm in Uncertainty Decision Making [J]. Computer Science, 2019, 46(6A): 593-597.
Full text



[1] WANG Huan, ZHANG Yun-feng and ZHANG Yan. Rapid Decision Method for Repairing Sequence Based on CFDs[J]. Computer Science, 2018, 45(3): 311 -316 .
[2] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[3] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[4] LIU Qin. Study on Data Quality Based on Constraint in Computer Forensics[J]. Computer Science, 2018, 45(4): 169 -172 .
[5] ZHONG Fei and YANG Bin. License Plate Detection Based on Principal Component Analysis Network[J]. Computer Science, 2018, 45(3): 268 -273 .
[6] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99, 116 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111, 142 .
[8] WANG Shuai, LIU Juan, BI Yao-yao, CHEN Zhe, ZHENG Qun-hua and DUAN Hui-fang. Automatic Recognition of Breast Gland Based on Two-step Clustering and Random Forest[J]. Computer Science, 2018, 45(3): 247 -252 .
[9] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121, 136 .
[10] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .