计算机科学 ›› 2019, Vol. 46 ›› Issue (3): 221-226.doi: 10.11896/j.issn.1002-137X.2019.03.033
袁丁,王茜,邓李维
YUAN Ding, WANG Qian, DENG Li-wei
摘要: 有监督深度学习在有大量标记数据的领域可以取得不错的效果,但实际上很多领域只有大量未标记的数据。如何利用大量无标记数据,成为了深度学习发展的一个关键问题,领域自适应就是解决这一问题的一种有效方法。目前,基于对抗训练的域适应方法取得了较好的效果,这类方法利用领域分类损失对源域和目标域的特征分布进行对齐,降低了两个领域特征表示的分布差异,使采用源域数据训练的模型可以应用在目标域数据上。现有的域适应方法是在适配后的特征上进行模型训练的,没有充分利用目标域数据的原始信息,当两个领域差异较大时,会降低目标域特征的域内可鉴别性。针对现有方法的弱点,文中基于对抗判别域适应方法(ADDA),提出了一种基于对目标域数据聚类辅助特征对齐的域适应方法(CAFA-DA)。该方法通过聚类获得目标域数据伪标记,并在域适应阶段约束特征编码器训练,利用目标域数据的原始信息提高目标域特征的可鉴别性;将聚类和域适应两个过程训练的分类器进行集成学习,用高置信度样本进行训练,以提升模型的最终效果。CAFA-DA可用在任何基于对抗损失的领域自适应方法上。最后,在领域自适应的4个标准数据集上将CAFA-DA方法与目前几种先进的领域自适应方法进行了对比实验,结果表明:CAFA-DA方法的实验结果比其他几种方法都好;相对于ADDA方法,该方法在2个对比实验上的准确率分别提升了3.2%和17.2%。
中图分类号:
[1]BEN-DAVID S,BLITZER J,CRAMMER K,et al.A theory of learning from different domains.Machine learning,2010,79(1-2):151-175. [2]TZENG E,HOFFMAN J,SAENKO K,et al.Adversarial dis- criminative domain adaptation[C]∥Computer Vision and Pattern Recognition (CVPR).IEEE,2017:4. [3]TZENG E,HOFFMAN J,ZHANG N,et al.Deep Domain Confusion:Maximizing for Domain Invariance.arXiv:1412.3474,2014. [4]LONG M,CAO Y,WANG J,et al.Learning transferable fea- tures with deep adaptation networks.arXiv:1502.02791,2015. [5]LONG M,ZHU H,WANG J,et al.Deep transfer learning with joint adaptation networks[C]∥Proceedings of the 34th International Conference on Machine Learning-Volume 70.2017:2208-2217. [6]GRETTON A,BORGWARDT K M,RASCH M J,et al.A kernel two-sample test.Journal of Machine Learning Research,2012,13(1):723-773. [7]GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adver- sarial training of neural networks.The Journal of Machine Learning Research,2016,17(1):2096-2030. [8]BOUSMALIS K,TRIGEORGIS G,SILBERMAN N,et al.Domain separation networks[C]∥Advances in Neural Information Processing Systems.IEEE,2016:343-351. [9]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Gene- rative adversarial nets[C]∥Advances in Neural Information Processing Systems.IEEE,2014:2672-2680. [10]GULRAJANI I,AHMED F,ARJOVSKY M,et al.Improved training of wasserstein gans[C]∥Advances in Neural Information Processing Systems.IEEE,2017:5767-5777. [11]SHU R,BUI H H,NARUI H,et al.A DIRT-T Approach to Unsupervised Domain Adaptation.arXiv:1802.08735,2018. [12]TAKERU M,SHIN-ICHI M,SHIN I,et al.Virtual Adversarial Training:A Regularization Method for Supervised and Semi-Supervised Learning.IEEE Transactions on Pattern Analysis and Machine Intelligence,2018:1. [13]SAITO K,USHIKU Y,HARADA T.Asymmetric tri-training for unsupervised domain adaptation.arXiv:1702.08400,2017. [14]CHEN C,XIE W,XU T,et al.Progressive Feature Alignment for Unsupervised Domain Adaptation.arXiv:1811.08585,2018. [15]KUMAR A,SATTIGERI P,WADHAWAN K,et al.Co-regularized Alignment for Unsupervised Domain Adaptation[C]∥Advances in Neural Information Processing Systems.IEEE,2018:9367-9378. [16]GRANDVALET Y,BENGIO Y.Semi-supervised learning by entropy minimization[C]∥Advances in Neural Information Processing Systems.IEEE,2005:529-536. [17]SHIMODAIRA H.Improving predictive inference under cova- riate shift by weighting the log-likelihood function.Journal of statistical planning and inference,2000,90(2):227-244. [18]MANSOUR Y,MOHRI M,ROSTAMIZADEH A.Domain ada- ptation:Learning bounds and algorithms.arXiv:0902.3430,2009. [19]CHEN X,DUAN Y,HOUTHOOFT R,et al.Infogan:Interpretable representation learning by information maximizingge-nerative adversarial nets[C]∥Advances in Neural Information Processing Systems.IEEE,2016:2172-2180. [20]TZENG E,HOFFMAN J,DARRELL T,et al.Simultaneous deep transfer across domains and tasks[C]∥Proceedings of the IEEE International Conference on Computer Vision.IEEE,2015:4068-4076. [21]GANIN Y,LEMPITSKY V.Unsupervised domain adaptation by backpropagation.arXiv:1409.7495,2014. [22]GHIFARY M,KLEIJN W B,ZHANG M,et al.Deep reconstruction-classification networks for unsupervised domain adaptation[C]∥European Conference on Computer Vision.Cham:Springer,2016:597-613. [23]LIU M Y,TUZEL O.Coupled generative adversarial networks[C]∥Advances in Neural Information Processing Systems.IEEE,2016:469-477. [24]BOUSMALIS K,SILBERMAN N,DOHAN D,et al.Unsupervised pixel-level domain adaptation with generative adversarial networks[C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017:7. [25]TAIGMAN Y,POLYAK A,WOLF L.Unsupervised cross-do- main image generation.arXiv:1611.02200,2016. [26]CHAPELLE O,ZIEN A.Semi-Supervised Classification by Low Density Separation[C]∥AISTATS.2005:57-64. [27]DAI Z,YANG Z,YANG F,et al.Good semi-supervised learning that requires a bad gan[C]∥Advances in Neural Information Processing Systems.IEEE,2017:6510-6520. [28]ZHOU Z H,LI M.Tri-training:Exploiting unlabeled data using three classifiers.IEEE Transactions on knowledge and Data Engineering,2005,17(11):1529-1541. [29]CARON M,BOJANOWSKI P,JOULIN A,et al.Deep Clustering for Unsupervised Learning of Visual Features//Computer Vision-ECCV 2018.Cham:Springer,2018:139-156. [30]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet Classification with Deep Convolutional Neural Networks[C]∥Advances in Neural Information Processing Systems.IEEE,2012:1097-1105. [31]SRIVASTAVA N,HINTON G,KRIZHEVSKY A,et al.Dropout:a simple way to prevent neural networks from overfitting.The Journal of Machine Learning Research,2014,15(1):1929-1958. [32]JOHNSON J,DOUZE M,JGOU H.Billion-scale similarity search with gpus.arXiv:1702.08734,2017. [33]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition.Proceedings of the IEEE,1998,86(11):2278-2324. [34]ARBELAEZ P,MAIRE M,FOWLKES C,et al.Contour detection and hierarchical image segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(5):898-916. [35]NETZER Y,WANG T,COATES A,et al.Reading digits in na- tural images with unsupervised feature learning[C]∥NIPS Workshop on Deep Learning and Unsupervised Feature Lear-ning.2011:5. [36]WANG Y,WANG Q,LV Q F,et al.An Improved K-means Algorithm Based on Initial Clustering Center Optimization and Weighted Between Dimension.Journal of Chongqing University of Technology(Natural Science),2013,27(4):77-80.(in Chinese) 王越,王泉,吕奇峰,等. 基于初始聚类中心优化和维间加权的改进K-means算法. 重庆理工大学学报(自然科学),2013,27(4):77-80. |
[1] | 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩. 基于分层抽样优化的面向异构客户端的联邦学习 Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients 计算机科学, 2022, 49(9): 183-193. https://doi.org/10.11896/jsjkx.220500263 |
[2] | 柴慧敏, 张勇, 方敏. 基于特征相似度聚类的空中目标分群方法 Aerial Target Grouping Method Based on Feature Similarity Clustering 计算机科学, 2022, 49(9): 70-75. https://doi.org/10.11896/jsjkx.210800203 |
[3] | 刘丽, 李仁发. 医疗CPS协作网络控制策略优化 Control Strategy Optimization of Medical CPS Cooperative Network 计算机科学, 2022, 49(6A): 39-43. https://doi.org/10.11896/jsjkx.210300230 |
[4] | 林夕, 陈孜卓, 王中卿. 基于不平衡数据与集成学习的属性级情感分类 Aspect-level Sentiment Classification Based on Imbalanced Data and Ensemble Learning 计算机科学, 2022, 49(6A): 144-149. https://doi.org/10.11896/jsjkx.210500205 |
[5] | 康雁, 吴志伟, 寇勇奇, 张兰, 谢思宇, 李浩. 融合Bert和图卷积的深度集成学习软件需求分类 Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution 计算机科学, 2022, 49(6A): 150-158. https://doi.org/10.11896/jsjkx.210500065 |
[6] | 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩. 基于DBSCAN聚类的集群联邦学习方法 Clustered Federated Learning Methods Based on DBSCAN Clustering 计算机科学, 2022, 49(6A): 232-237. https://doi.org/10.11896/jsjkx.211100059 |
[7] | 郁舒昊, 周辉, 叶春杨, 王太正. SDFA:基于多特征融合的船舶轨迹聚类方法研究 SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion 计算机科学, 2022, 49(6A): 256-260. https://doi.org/10.11896/jsjkx.211100253 |
[8] | 毛森林, 夏镇, 耿新宇, 陈剑辉, 蒋宏霞. 基于密度敏感距离和模糊划分的改进FCM算法 FCM Algorithm Based on Density Sensitive Distance and Fuzzy Partition 计算机科学, 2022, 49(6A): 285-290. https://doi.org/10.11896/jsjkx.210700042 |
[9] | 陈景年. 一种适于多分类问题的支持向量机加速方法 Acceleration of SVM for Multi-class Classification 计算机科学, 2022, 49(6A): 297-300. https://doi.org/10.11896/jsjkx.210400149 |
[10] | 陈佳舟, 赵熠波, 徐阳辉, 马骥, 金灵枫, 秦绪佳. 三维城市场景中的小物体检测 Small Object Detection in 3D Urban Scenes 计算机科学, 2022, 49(6): 238-244. https://doi.org/10.11896/jsjkx.210400174 |
[11] | 王宇飞, 陈文. 基于DECORATE集成学习与置信度评估的Tri-training算法 Tri-training Algorithm Based on DECORATE Ensemble Learning and Credibility Assessment 计算机科学, 2022, 49(6): 127-133. https://doi.org/10.11896/jsjkx.211100043 |
[12] | 韩红旗, 冉亚鑫, 张运良, 桂婕, 高雄, 易梦琳. 基于共同子空间分类学习的跨媒体检索研究 Study on Cross-media Information Retrieval Based on Common Subspace Classification Learning 计算机科学, 2022, 49(5): 33-42. https://doi.org/10.11896/jsjkx.210200157 |
[13] | 邢云冰, 龙广玉, 胡春雨, 忽丽莎. 基于SVM的类别增量人体活动识别方法 Human Activity Recognition Method Based on Class Increment SVM 计算机科学, 2022, 49(5): 78-83. https://doi.org/10.11896/jsjkx.210400024 |
[14] | 朱哲清, 耿海军, 钱宇华. 面向化学结构的线段聚类算法 Line-Segment Clustering Algorithm for Chemical Structure 计算机科学, 2022, 49(5): 113-119. https://doi.org/10.11896/jsjkx.210700131 |
[15] | 张宇姣, 黄锐, 张福泉, 隋栋, 张虎. 基于菌群优化的近邻传播聚类算法研究 Study on Affinity Propagation Clustering Algorithm Based on Bacterial Flora Optimization 计算机科学, 2022, 49(5): 165-169. https://doi.org/10.11896/jsjkx.210800218 |
|