基于张量分解的域适应算法

doi:10.11896/jsjkx.190300095

摘要/Abstract

摘要： 由于训练数据易过期,在多数情况下训练数据和测试数据具有不同的特征分布,因此在利用源域信息时,须先尽量减小不同领域的特征分布的差异。使用张量表示特征可以维持高维空间数据的本征结构信息。朴素张量子空间学习法虽然是面向张量特征的域适应方法,但其复杂度较高,且没有达到较好的知识迁移效果。为此,文中提出了基于张量分解的域适应算法,即张量列子空间学习法和张量环子空间学习法,二者的主要思想相似。首先,使用张量表示源域和目标域的特征;其次利用张量分解方法,将特征分解为一系列三阶张量来表示子空间;然后,依次将源域特征和目标域特征映射到子空间中;最后,将特征张量重塑为矩阵形式,基于映射后的源域特征训练模型,基于映射后的目标域特征完成新领域的任务。实验结果表明,在无监督图像分类中,张量列子空间学习法和张量环子空间学习法在准确率和运行时间方面都有所提升。相比于朴素张量子空间学习法,张量列子空间学习法和张量环子空间学习法的准确率分别提高了1.68%和2.08%,且运行时间也有明显减少,算法复杂度较小。实验数据充分说明,基于张量分解的域适应算法充分减小了源域特征和目标域特征之间的差异,实现了不同领域间的知识复用。

关键词: 迁移学习, 图像分类, 域适应, 张量分解, 子空间学习

Abstract: Because training data tend to be outdated,training data and test data have different feature distributions in most cases.Therefore,when using the source domain information,it is necessary to minimize the difference of feature distributions in different fields.Features represented by tensor can maintain the intrinsic structure information of high-dimensional spatial data.Naive tensor subspace learning is a domain adaptation method for tensor features,but it has high complexity and can not achieve good knowledge transfer effect.For this reason,this paper proposed a domain adaptation algorithm based on tensor decomposition,namely tensor train subspace learning and tensor ring subspace lear-ning,and the main ideas of the two methods are similar.Firstly,the features of source domain and target domain are coded into tensor.By using the tensor decomposition,the tensor of features is decomposed into a series of third-order tensors to represent the subspace.Then,the features of source domain and target domain are mapped into subspace successively.Finally,the feature tensor is reshaped into matrix form.Based on the mapped features of source domain training model and the mapped feature of target domain,the task in new domain is completed.Experiments show that the tensor train subspace learning and the tensor ring subspace learning are improved in terms of accuracy and running time for unsupervised image classification.Compared with the naive tensor subspace learning,the accuracy of the tensor train subspace learning and the tensor ring subspace learning is improved by 1.68% and 2.08% respectively,the running time is also reduced significantly,and the complexity of the algorithm is smaller.Experimental results show that the domain adaptation algorithm based on tensor decomposition can reduce the difference between source domain and target domain,and realize the reuse of knowledge between different domains.

Key words: Domain adaptation, Image classification, Subspace learning, Tensor decomposition, Transfer learning

中图分类号:

TP181

徐书艳, 韩立新, 徐国夏. 基于张量分解的域适应算法[J]. 计算机科学, 2019, 46(12): 89-94. https://doi.org/10.11896/jsjkx.190300095

XU Shu-yan, HAN Li-xin, XU Guo-xia. Domain Adaptation Algorithm Based on Tensor Decomposition[J]. Computer Science, 2019, 46(12): 89-94. https://doi.org/10.11896/jsjkx.190300095

参考文献

[1]PAN S J,YANG Q.A survey on transfer learning[J].IEEE Transactions on knowledge and data engineering,2010,22(10):1345-1359.
[2]LONG M S.Transfer Learning:Problems and Methods[D].Beijing:Tsinghua University,2014.(in Chinese)
龙明盛.迁移学习问题与方法研究[D].北京:清华大学,2014.
[3]ZHANG Z,WANG M,HUANG Y,et al.Aligning infinite-dimensional covariance matrices in reproducing kernel hilbert spaces for domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:3437-3445.
[4]YANG B,MA A J,YUEN P C.Domain-Shared Group-Sparse Dictionary Learning for Unsupervised Domain Adaptation[C]//Thirty-Second AAAI Conference on Artificial Intelligence.AAAI,2018.
[5]MENG J,HU G Y,PAN Z S,et al.Research and Perspective on Domain Adaptation Learning Algorithms[J].Computer Science,2015,42(10):7-12,34.(in Chinese)
孟娟,胡谷雨,潘志松,等.领域适应学习算法研究与展望[J].计算机科学,2015,42(10):7-12,34.
[6]LIU J W,SUN Z K,LUO X L.Review and Research Development on Domain Adaptation Learning[J].Acta Automatica Sinica,2014,40(8):1576-1600.(in Chinese)
刘建伟,孙正康,罗雄麟.域自适应学习研究进展[J].自动化学报,2014,40(8):1576-1600.
[7]LU H,ZHANG L,CAO Z,et al.When unsupervised domain adaptation meets tensor representations[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2017:599-608.
[8]TUCKER L R.Some mathematical notes on three-mode factor analysis[J].Psychometrika,1966,31(3):279-311.
[9]KOLDA T G,BADER B W.Tensor decompositions and applications[J].SIAM review,2009,51(3):455-500.
[10]ORÚ S R.A practical introduction to tensor networks:Matrix product states and projected entangled pair states[J].Annals of Physics,2014,349:117-158.
[11]MURG V,VERSTRAETE F,SCHNEIDER R,et al.Tree tensor network state with variable tensor order:An efficient multireference method for strongly correlated systems[J].Journal of chemical theory and computation,2015,11(3):1027-1036.
[12]OSELEDETS I V.Tensor-train decompositi-on[J].SIAM Journal on Scientific Computing,2011,33(5):2295-2317.
[13]HÜBENER R,NEBENDAHL V,DÚ R W.Concatenated tensor network states[J].New Journal of Physics,2010,12(2):025004.
[14]BENGUA J A,HO P N,TUAN H D,et al.Matrix product state for higher-order tensor compression and classification[J].IEEE Transactions on Signal Processing,2017,65(15):4019-4030.
[15]ZHAO Q,ZHOU G,XIE S,et al.Tensor ring decomposition [J].arXiv:1606.05535,2016.
[16]MICKELIN O,KARAMAN S.Tensor ring decomposition[J].arXiv:1807.02513,2018.
[17]WANG W,AGGARWAL V,AERON S.Efficient low rank tensor ring completion[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2017:5697-5705.
[18]FERNANDO B,HABRARD A,SEBBAN M,et al.Unsuper- vised visual domain adaptation using subspace alignment[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2013:2960-2967.
[19]GONG B,SHI Y,SHA F,et al.Geodesic flow kernel for unsupervised domain adaptationn[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2012:2066-2073.
[20]LONG M,WANG J,DING G,et al.Transfer learning with graph co-regularization[J].IEEE Transactions on Knowledge and Data Engineering,2014,26(7):1805-1818.
[21]SAENKO K,KULIS B,FRITZ M,et al.Adapting visual category models to new domains[C]//European Conference on Computer Vision.Berlin:Springer,2010:213-226.
[22]GRIFFIN G,HOLUB A,PERONA P.Caltech-256 object category dataset:Technical Report 7694 [R].California Institute of Technology,2007.
[23]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[24]HE K,ZHANG X,REN S,et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.

相关文章 15

[1]	武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航. 监督和半监督学习下的多标签分类综述 Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning 计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111
[2]	方义秋, 张震坤, 葛君伟. 基于自注意力机制和迁移学习的跨领域推荐算法 Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning 计算机科学, 2022, 49(8): 70-77. https://doi.org/10.11896/jsjkx.210600011
[3]	杜丽君, 唐玺璐, 周娇, 陈玉兰, 程建. 基于注意力机制和多任务学习的阿尔茨海默症分类 Alzheimer's Disease Classification Method Based on Attention Mechanism and Multi-task Learning 计算机科学, 2022, 49(6A): 60-65. https://doi.org/10.11896/jsjkx.201200072
[4]	王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰. 基于多源迁移学习的大坝裂缝检测 Dam Crack Detection Based on Multi-source Transfer Learning 计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[5]	杨健楠, 张帆. 一种结合双注意力机制和层次网络结构的细碎农作物分类方法 Classification Method for Small Crops Combining Dual Attention Mechanisms and Hierarchical Network Structure 计算机科学, 2022, 49(6A): 353-357. https://doi.org/10.11896/jsjkx.210200169
[6]	朱旭东, 熊贇. 基于样本分布损失的图像多标签分类研究 Study on Multi-label Image Classification Based on Sample Distribution Loss 计算机科学, 2022, 49(6): 210-216. https://doi.org/10.11896/jsjkx.210300267
[7]	张文轩, 吴秦. 基于多分支注意力增强的细粒度图像分类 Fine-grained Image Classification Based on Multi-branch Attention-augmentation 计算机科学, 2022, 49(5): 105-112. https://doi.org/10.11896/jsjkx.210100108
[8]	彭云聪, 秦小林, 张力戈, 顾勇翔. 面向图像分类的小样本学习算法综述 Survey on Few-shot Learning Algorithms for Image Classification 计算机科学, 2022, 49(5): 1-9. https://doi.org/10.11896/jsjkx.210500128
[9]	谭珍琼, 姜文君, 任演纳, 张吉, 任德盛, 李晓鸿. 基于二分图的个性化学习任务分配 Personalized Learning Task Assignment Based on Bipartite Graph 计算机科学, 2022, 49(4): 269-281. https://doi.org/10.11896/jsjkx.210500125
[10]	左杰格, 柳晓鸣, 蔡兵. 基于图像分块与特征融合的户外图像天气识别 Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion 计算机科学, 2022, 49(3): 197-203. https://doi.org/10.11896/jsjkx.201200263
[11]	张舒萌, 余增, 李天瑞. 跨领域文本的可迁移情绪分析方法 Transferable Emotion Analysis Method for Cross-domain Text 计算机科学, 2022, 49(3): 218-224. https://doi.org/10.11896/jsjkx.210400034
[12]	许华杰, 陈育, 杨洋, 秦远卓. 基于混合样本自动数据增强技术的半监督学习方法 Semi-supervised Learning Method Based on Automated Mixed Sample Data Augmentation Techniques 计算机科学, 2022, 49(3): 288-293. https://doi.org/10.11896/jsjkx.210100156
[13]	李星燃, 张立言, 姚树婧. 结合特征融合和注意力机制的微表情识别方法 Micro-expression Recognition Method Combining Feature Fusion and Attention Mechanism 计算机科学, 2022, 49(2): 4-11. https://doi.org/10.11896/jsjkx.210900028
[14]	董琳, 黄丽清, 叶锋, 黄添强, 翁彬, 徐超. 人脸伪造检测泛化性方法综述 Survey on Generalization Methods of Face Forgery Detection 计算机科学, 2022, 49(2): 12-30. https://doi.org/10.11896/jsjkx.210900146
[15]	宁秋怡, 史小静, 段湘煜, 张民. 基于风格感知的无监督领域适应算法 Unsupervised Domain Adaptation Based on Style Aware 计算机科学, 2022, 49(1): 271-278. https://doi.org/10.11896/jsjkx.201200094

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed