计算机科学 ›› 2020, Vol. 47 ›› Issue (8): 151-156.doi: 10.11896/jsjkx.190600175

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于PE散度实例过滤的深度域适应方法

袁晨晖, 程春玲   

  1. 南京邮电大学计算机学院、软件学院、网络空间安全学院 南京 210023
  • 出版日期:2020-08-15 发布日期:2020-08-10
  • 通讯作者: 程春玲(chengcl@njupt.edu.cn)
  • 作者简介:958138791@qq.com
  • 基金资助:
    国家自然科学基金项目(61672297)

Deep Domain Adaptation Algorithm Based on PE Divergence Instance Filtering

YUAN Chen-hui, CHENG Chun-ling   

  1. College of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210023, China
  • Online:2020-08-15 Published:2020-08-10
  • About author:YUAN Chen-hui, born in 1994, postgraduate.Her main research interests include deep learning and transfer learning.
    CHENG Chun-ling, born in 1972, professor, is a member of China Computer Federation.Her main research interests include data management, data warehousing and data mining.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61672297).

摘要: 深度域适应作为迁移学习最常见的问题之一, 已经在许多机器学习应用中获得了优异的性能。然而, 现有的深度域适应方法在减小域偏差时单一适配完全连接层, 忽视了卷积层的空间信息和语义上下文信息, 造成在知识迁移过程中丢失重要信息。为此, 文中将基于实例的域适应与基于特征的域适应相结合, 提出了基于PE散度实例过滤的深度域适应方法(Domain Adaptation Based on PE Divergence Instance Filtering, DAPEIF)。其基本思想是首先利用PE散度计算源域样本的相对权值, 删除易造成负迁移的源域样本, 选择相对权值较高的训练数据作为新的源域样本, 从而降低源域与目标域之间的差异性;然后基于AlexNet模型, 使用最大均值差异(Maximum Mean Discrepancy, MMD)准则, 将其作为正则化项纳入神经网络的学习中。与以往只关注完全连接层的域适应方法不同, 文中联合匹配卷积层和完全连接层的边缘概率分布以解决欠适配问题, 同时引入权值正则项, 通过梯度下降法学习网络参数, 进一步提高了域适应过程中模型的泛化性能。所提算法能同时对神经网络的卷积层和完全连接层的参数赋予领域自适应能力, 在域适应公开数据集Office-Caltech上的实验结果表明, 域适应对卷积层是有效的, 更重要的是卷积层的自适应可以进一步提高传统域适应的性能, 与主流的域适应算法DDC和DAN相比, 所提算法在精度上有所提高, 平均精度达到84.46%。

关键词: PE散度, 迁移学习, 深度域适应, 域偏差, 最大均值差异

Abstract: Deep domain adaptation, one of the most common problems in transfer learning, has achieved excellent performance in many machine learning applications.However, the existing deep domain adaptation just adapts the fully connected layer while reduce domain shift, which ignores the spatial information and semantic context information of the convolutional layer, resulting in the loss of important information during the process of knowledge transfer.To this end, this paper combines instance-based domain adaptation with feature-based domain adaptation and proposes a deep domain adaptation algorithm based on PE divergence instance filtering (DAPEIF).Firstly, the relative importance weight of the source samples is calculated by PE divergence, and the source samples that are likely to cause negative transfer are deleted.Then the maximum mean discrepancy metric is included as a regularization term in the training of the neural network based on AlexNet, and the marginal probability distribution of the convolutional layer and the fully connected layer are jointly adapted to reduce the difference between the new source domain and the target domain.At the same time, the weight regularization term is introduced, and the network parameters are learned by the gradient descent to further improve the generalization performance of the model in the process of domain adaptation.The proposed algorithm can simultaneously assign domain adaptative ability to the parameters of the convolutional layer and the fully connected layer of the neural network.The experimental results on the public dataset Office-Caltech in domain adaptation demonstrate that compared with the state of the art domain adaptation algorithms, the proposed algorithm has better performance in accuracy and the average accuracy reaches 84.46%.

Key words: Domain adaptation, Domain shift, Maximum mean discrepancy, Pearson divergence, Transfer learning

中图分类号: 

  • TP311
[1]ZHUANG F Z, LUO P, HE Q, et al.Survey on transfer learning research[J].Journal of Software, 2015, 26(1):26-29.
[2]SHANKAR S, PIRATLA V, CHAKRABARTI S, et al.Generalizing Across Domains via Cross-Gradient Training[C]∥International Conference on Learning Representations (ICLR).2018.
[3]PENG P, XIANG T, WANG Y, et al.Unsupervised cross-dataset transfer learning for person re-identification[C]∥IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2018:1306-1315.
[4]GRETTON A, BORGWARDT K M, RASCH M, et al.A kernel method for the two-sample-problem[C]∥Proceedings of Advances in Neural Information Processing Systems.Vancouver, Canada:MIT Press, 2006:513-520.
[5]PAN S J, TSANG I W, KWOK J T, et al.Domain adaptation via transfer component analysis[J].IEEE Transactions on Neural Networks, 2011, 22(2):199-210.
[6]LONG M, WANG J, DING G, et al.Transfer Joint Matching for Unsupervised Domain Adaptation[C]∥IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE, 2014:1410-1417.
[7]LONG M, WANG J, DING G, et al.Transfer Feature Learning with Joint Distribution Adaptation[C]∥Proceedings of the 2013 IEEE International Conference on Computer Vision.2013:2000-2007.
[8]SHA F, SHI Y, GONG B, et al.Geodesic flow kernel for unsupervised domain adaptation[C]∥IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society, 2015:2066-2073.
[9]LU H, SHEN C, CAO Z, et al.An Embarrassingly Simple Approach to Visual Domain Adaptation[J].IEEE Transactions on Image Processing, 2018, 27(7):3403-3417.

[10]LI J, LU K, HUANG Z, et al.Transfer Independently Toge-ther:A Generalized Framework for Domain Adaptation[J].IEEE Transactions on Cybernetics, 2018:1-12.
[11]GHIFARY M, KLEIJN W B, ZHANG M.Domain adaptive neural networks for object recognition[J].Technical report, arXiv:1409.6041, 2014.
[12]TZENG E, HOFFMAN J, ZHANG N, et al.Deep Domain Confusion:Maximizing for Domain Invariance[J].arXiv:1412.3474, 2014.
[13]LONG M, CAO Y, WANG J, et al.Learning transferable fea-tures with deep adaptation networks[C]∥International Confe-rence on Machine Learning.2015:97-105.
[14]PEARSON K.On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that in can reasonably be supposed to have arisen from random sampling[J].Philosoph Magazine, 1992, 50(302):157-175.
[15]KRIZHEVSKY A, SUTSKEVER I, HINTON G.ImageNetClassification with Deep Convolutional Neural Networks[J].Advances in neural information processing systems, 2012, 25(2):1097-1105.
[16]LONG M, WANG J, DING G, et al.Adaptation Regularization:A General Framework for Transfer Learning[J].IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5):1076-1089.
[17]SAENKO K, KULIS B, FRITZ M, et al.Adapting visual category models to new domains[J].Computer Vision ECCV, 2010, 63(14):213-226.
[1] 方义秋, 张震坤, 葛君伟.
基于自注意力机制和迁移学习的跨领域推荐算法
Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning
计算机科学, 2022, 49(8): 70-77. https://doi.org/10.11896/jsjkx.210600011
[2] 王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰.
基于多源迁移学习的大坝裂缝检测
Dam Crack Detection Based on Multi-source Transfer Learning
计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[3] 彭云聪, 秦小林, 张力戈, 顾勇翔.
面向图像分类的小样本学习算法综述
Survey on Few-shot Learning Algorithms for Image Classification
计算机科学, 2022, 49(5): 1-9. https://doi.org/10.11896/jsjkx.210500128
[4] 谭珍琼, 姜文君, 任演纳, 张吉, 任德盛, 李晓鸿.
基于二分图的个性化学习任务分配
Personalized Learning Task Assignment Based on Bipartite Graph
计算机科学, 2022, 49(4): 269-281. https://doi.org/10.11896/jsjkx.210500125
[5] 左杰格, 柳晓鸣, 蔡兵.
基于图像分块与特征融合的户外图像天气识别
Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion
计算机科学, 2022, 49(3): 197-203. https://doi.org/10.11896/jsjkx.201200263
[6] 张舒萌, 余增, 李天瑞.
跨领域文本的可迁移情绪分析方法
Transferable Emotion Analysis Method for Cross-domain Text
计算机科学, 2022, 49(3): 218-224. https://doi.org/10.11896/jsjkx.210400034
[7] 李星燃, 张立言, 姚树婧.
结合特征融合和注意力机制的微表情识别方法
Micro-expression Recognition Method Combining Feature Fusion and Attention Mechanism
计算机科学, 2022, 49(2): 4-11. https://doi.org/10.11896/jsjkx.210900028
[8] 侯宏旭, 孙硕, 乌尼尔.
蒙汉神经机器翻译研究综述
Survey of Mongolian-Chinese Neural Machine Translation
计算机科学, 2022, 49(1): 31-40. https://doi.org/10.11896/jsjkx.210900006
[9] 熊朝阳, 王婷.
基于卷积神经网络的建筑构件图像识别
Image Recognition for Building Components Based on Convolutional Neural Network
计算机科学, 2021, 48(6A): 51-56. https://doi.org/10.11896/jsjkx.200500122
[10] 吴兰, 王涵, 李斌全.
基于自监督任务最优选择的无监督域自适应方法
Unsupervised Domain Adaptive Method Based on Optimal Selection of Self-supervised Tasks
计算机科学, 2021, 48(6A): 357-363. https://doi.org/10.11896/jsjkx.201000030
[11] 李达, 雷迎科, 张海川.
基于LTE网络的室外指纹定位
Outdoor Fingerprint Positioning Based on LTE Networks
计算机科学, 2021, 48(6A): 404-409. https://doi.org/10.11896/jsjkx.200700170
[12] 刘昱彤, 李鹏, 孙云云, 胡素君.
基于深度动态联合自适应网络的图像识别方法
Image Recognition with Deep Dynamic Joint Adaptation Networks
计算机科学, 2021, 48(6): 131-137. https://doi.org/10.11896/jsjkx.210100008
[13] 马闯, 田青, 孙赫阳, 曹猛, 马廷淮.
基于双重权重偏差建模的无监督域适应
Unsupervised Domain Adaptation Based on Weighting Dual Biases
计算机科学, 2021, 48(2): 217-223. https://doi.org/10.11896/jsjkx.200700028
[14] 周彦, 陈少昌, 吴可, 宁明强, 陈宏昆, 张鹏.
SCTD1.0:声呐常见目标检测数据集
SCTD 1.0:Sonar Common Target Detection Dataset
计算机科学, 2021, 48(11A): 334-339. https://doi.org/10.11896/jsjkx.210100138
[15] 刘林芽, 吴送英, 左志远, 曹子文.
基于YOLOv3算法的山区铁路边坡落石检测方法研究
Research on Rockfall Detection Method of Mountain Railway Slope Based on YOLOv3 Algorithm
计算机科学, 2021, 48(11A): 290-294. https://doi.org/10.11896/jsjkx.201200113
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!