Computer Science ›› 2020, Vol. 47 ›› Issue (8): 151-156.doi: 10.11896/jsjkx.190600175

Previous Articles     Next Articles

Deep Domain Adaptation Algorithm Based on PE Divergence Instance Filtering

YUAN Chen-hui, CHENG Chun-ling   

  1. College of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210023, China
  • Online:2020-08-15 Published:2020-08-10
  • About author:YUAN Chen-hui, born in 1994, postgraduate.Her main research interests include deep learning and transfer learning.
    CHENG Chun-ling, born in 1972, professor, is a member of China Computer Federation.Her main research interests include data management, data warehousing and data mining.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61672297).

Abstract: Deep domain adaptation, one of the most common problems in transfer learning, has achieved excellent performance in many machine learning applications.However, the existing deep domain adaptation just adapts the fully connected layer while reduce domain shift, which ignores the spatial information and semantic context information of the convolutional layer, resulting in the loss of important information during the process of knowledge transfer.To this end, this paper combines instance-based domain adaptation with feature-based domain adaptation and proposes a deep domain adaptation algorithm based on PE divergence instance filtering (DAPEIF).Firstly, the relative importance weight of the source samples is calculated by PE divergence, and the source samples that are likely to cause negative transfer are deleted.Then the maximum mean discrepancy metric is included as a regularization term in the training of the neural network based on AlexNet, and the marginal probability distribution of the convolutional layer and the fully connected layer are jointly adapted to reduce the difference between the new source domain and the target domain.At the same time, the weight regularization term is introduced, and the network parameters are learned by the gradient descent to further improve the generalization performance of the model in the process of domain adaptation.The proposed algorithm can simultaneously assign domain adaptative ability to the parameters of the convolutional layer and the fully connected layer of the neural network.The experimental results on the public dataset Office-Caltech in domain adaptation demonstrate that compared with the state of the art domain adaptation algorithms, the proposed algorithm has better performance in accuracy and the average accuracy reaches 84.46%.

Key words: Domain adaptation, Domain shift, Maximum mean discrepancy, Pearson divergence, Transfer learning

CLC Number: 

  • TP311
[1]ZHUANG F Z, LUO P, HE Q, et al.Survey on transfer learning research[J].Journal of Software, 2015, 26(1):26-29.
[2]SHANKAR S, PIRATLA V, CHAKRABARTI S, et al.Generalizing Across Domains via Cross-Gradient Training[C]∥International Conference on Learning Representations (ICLR).2018.
[3]PENG P, XIANG T, WANG Y, et al.Unsupervised cross-dataset transfer learning for person re-identification[C]∥IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2018:1306-1315.
[4]GRETTON A, BORGWARDT K M, RASCH M, et al.A kernel method for the two-sample-problem[C]∥Proceedings of Advances in Neural Information Processing Systems.Vancouver, Canada:MIT Press, 2006:513-520.
[5]PAN S J, TSANG I W, KWOK J T, et al.Domain adaptation via transfer component analysis[J].IEEE Transactions on Neural Networks, 2011, 22(2):199-210.
[6]LONG M, WANG J, DING G, et al.Transfer Joint Matching for Unsupervised Domain Adaptation[C]∥IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE, 2014:1410-1417.
[7]LONG M, WANG J, DING G, et al.Transfer Feature Learning with Joint Distribution Adaptation[C]∥Proceedings of the 2013 IEEE International Conference on Computer Vision.2013:2000-2007.
[8]SHA F, SHI Y, GONG B, et al.Geodesic flow kernel for unsupervised domain adaptation[C]∥IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society, 2015:2066-2073.
[9]LU H, SHEN C, CAO Z, et al.An Embarrassingly Simple Approach to Visual Domain Adaptation[J].IEEE Transactions on Image Processing, 2018, 27(7):3403-3417.

[10]LI J, LU K, HUANG Z, et al.Transfer Independently Toge-ther:A Generalized Framework for Domain Adaptation[J].IEEE Transactions on Cybernetics, 2018:1-12.
[11]GHIFARY M, KLEIJN W B, ZHANG M.Domain adaptive neural networks for object recognition[J].Technical report, arXiv:1409.6041, 2014.
[12]TZENG E, HOFFMAN J, ZHANG N, et al.Deep Domain Confusion:Maximizing for Domain Invariance[J].arXiv:1412.3474, 2014.
[13]LONG M, CAO Y, WANG J, et al.Learning transferable fea-tures with deep adaptation networks[C]∥International Confe-rence on Machine Learning.2015:97-105.
[14]PEARSON K.On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that in can reasonably be supposed to have arisen from random sampling[J].Philosoph Magazine, 1992, 50(302):157-175.
[15]KRIZHEVSKY A, SUTSKEVER I, HINTON G.ImageNetClassification with Deep Convolutional Neural Networks[J].Advances in neural information processing systems, 2012, 25(2):1097-1105.
[16]LONG M, WANG J, DING G, et al.Adaptation Regularization:A General Framework for Transfer Learning[J].IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5):1076-1089.
[17]SAENKO K, KULIS B, FRITZ M, et al.Adapting visual category models to new domains[J].Computer Vision ECCV, 2010, 63(14):213-226.
[1] FANG Yi-qiu, ZHANG Zhen-kun, GE Jun-wei. Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning [J]. Computer Science, 2022, 49(8): 70-77.
[2] WANG Jun-feng, LIU Fan, YANG Sai, LYU Tan-yue, CHEN Zhi-yu, XU Feng. Dam Crack Detection Based on Multi-source Transfer Learning [J]. Computer Science, 2022, 49(6A): 319-324.
[3] PENG Yun-cong, QIN Xiao-lin, ZHANG Li-ge, GU Yong-xiang. Survey on Few-shot Learning Algorithms for Image Classification [J]. Computer Science, 2022, 49(5): 1-9.
[4] TAN Zhen-qiong, JIANG Wen-Jun, YUM Yen-na-cherry, ZHANG Ji, YUM Peter-tak-shing, LI Xiao-hong. Personalized Learning Task Assignment Based on Bipartite Graph [J]. Computer Science, 2022, 49(4): 269-281.
[5] ZUO Jie-ge, LIU Xiao-ming, CAI Bing. Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion [J]. Computer Science, 2022, 49(3): 197-203.
[6] ZHANG Shu-meng, YU Zeng, LI Tian-rui. Transferable Emotion Analysis Method for Cross-domain Text [J]. Computer Science, 2022, 49(3): 218-224.
[7] NING Qiu-yi, SHI Xiao-jing, DUAN Xiang-yu, ZHANG Min. Unsupervised Domain Adaptation Based on Style Aware [J]. Computer Science, 2022, 49(1): 271-278.
[8] LIU Kai, ZHANG Hong-jun, CHEN Fei-qiong. Name Entity Recognition for Military Based on Domain Adaptive Embedding [J]. Computer Science, 2022, 49(1): 292-297.
[9] WU Lan, WANG Han, LI Bin-quan. Unsupervised Domain Adaptive Method Based on Optimal Selection of Self-supervised Tasks [J]. Computer Science, 2021, 48(6A): 357-363.
[10] LI Da, LEI Ying-ke, ZHANG Hai-chuan. Outdoor Fingerprint Positioning Based on LTE Networks [J]. Computer Science, 2021, 48(6A): 404-409.
[11] XIONG Zhao-yang, WANG Ting. Image Recognition for Building Components Based on Convolutional Neural Network [J]. Computer Science, 2021, 48(6A): 51-56.
[12] LIU Yu-tong, LI Peng, SUN Yun-yun, HU Su-jun. Image Recognition with Deep Dynamic Joint Adaptation Networks [J]. Computer Science, 2021, 48(6): 131-137.
[13] MA Chuang, TIAN Qing, SUN He-yang, CAO Meng, MA Ting-huai. Unsupervised Domain Adaptation Based on Weighting Dual Biases [J]. Computer Science, 2021, 48(2): 217-223.
[14] LIU Lin-ya, WU Song-ying, ZUO Zhi-yuan, CAO Zi-wen. Research on Rockfall Detection Method of Mountain Railway Slope Based on YOLOv3 Algorithm [J]. Computer Science, 2021, 48(11A): 290-294.
[15] ZHOU Yan, CHEN Shao-chang, WU Ke, NING Ming-qiang, CHEN Hong-kun, ZHANG Peng. SCTD 1.0:Sonar Common Target Detection Dataset [J]. Computer Science, 2021, 48(11A): 334-339.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!