计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (11A): 230100068-8.doi: 10.11896/jsjkx.230100068

• 图像处理&多媒体技术 • 上一篇    下一篇

基于任务关联特征解耦网络的无监督领域自适应图像分类

唐珺琨1, 张辉2, 张邹铨1, 吴天月1   

  1. 1 长沙理工大学电气与信息工程学院 长沙 410000
    2 湖南大学机器人学院 长沙 410000
  • 发布日期:2023-11-09
  • 通讯作者: 张辉(zhanghuihby@126.com)
  • 作者简介:(jadekintang@126.com)
  • 基金资助:
    国家重大研究计划-重点支持项目(92148204);国家重点研发计划(2021ZD0114503);国家自然科学基金(61971071,62027810,62133005);湖南省杰出青年科学基金项目(2021JJ10025);湖南省重点研发计划(2021GK4011,2022GK2011);长沙市科技重大专项(kh2003026);机器人学国家重点实验室联合开放基金(2021-KF-22-17);中国高校产学研创新基金(2020HYA06006);湖南科技创新领军人才项目(2022RC3063)

Image Classification for Unsupervised Domain Adaptation Based on Task Relevant FeatureSeparation Network

TANG Junkun1, ZHANG Hui2, ZHANG Zhouquan1and WU Tianyue1   

  1. 1 School of Electrical & Information Engineering,Changsha University of Science & Technology,Changsha 410000,China
    2 School of Robotics,Hunan University,Changsha 410000,China
  • Published:2023-11-09
  • About author:TANG Junkun,born in 1997,postgra-ducate.His main research interests include image processing,deep learning,transfer learning,and domain adaptation.
    ZHANG Hui,born in 1983,Ph.D,professor.His main research interests include machine vision,image processing,and robot vision.
  • Supported by:
    Major Research Plan of the National Natural Science Foundation of China(92148204),National Key R&D Program of China(2021ZD0114503),National Natural Science Foundation of China(61971071,62027810,62133005),Hunan Science Fund for Distinguished Young Scholars(2021JJ10025),Hunan Key Research and Development Program(2021GK4011,2022GK2011),Changsha Science and Technology Major Project(kh2003026),Joint Open Foundation of State Key Laboratory of Robotics(2021-KF-22-17),China University Industry-University-research Innovation Fund(2020HYA06006) and Hunan Leading Talent of Technological Innovation(2022RC3063).

PDF (PC)

237

摘要： 无监督领域自适应(Unsupervised Domain Adaptation,UDA)旨在帮助模型在跨域分布差异条件下从带标注的源域中学习到知识,以迁移至无标注的目标域。当前先进的域自适应方法主要通过直接对目标域与源域分布对齐来实现,其中特征往往被当作一个整体对象用于开展域间自适应任务,忽略了特征中的任务关联信息(域间不变、域内独特信息)与无关信息(颜色对比度、图像风格)耦合的情况,使得模型难以把握关键的特征信息,从而导致次优化。针对上述问题,提出了一种基于任务关联特征解耦网络的无监督领域自适应分类方法(Task Relevant Feature Separation Network,TRFS),通过对域间风格混合干扰下的特征与原始特征的注意力进行一致性的学习,来帮助网络提炼出与下游任务相关的特征权重,并进一步采用权重差获取任务无关特征权重,而后通过正交函数约束推远任务关联与无关特征,实现特征解耦;设计了任务特征细化解耦层,减轻配对特征与域独特特征混淆的情形,优化模型对分类判别的精度。此外,为了提升伪标签质量,引入基于记忆力银行的领域聚合伪标签生成方法,用于降低伪标签噪声。综合实验结果表明,所设计解耦模块具有良好的即插即用性,能够提升自适应方法的性能;且所提方法相比其他先进的域适应方法具有明显的优势,其中在Office-Home数据集上达到了73.6%的分类精度。

关键词: 特征解耦, 任务关联, 注意力机制, 无监督领域自适应, 图像分类

Abstract: Unsupervised domain adaptation(UDA) aims to assist the model in transferring learned information from a labeled source domain to an unlabeled target domain,given cross-domain distribution discrepancies.Current advanced domain adaptation techniques rely mostly on aligning the distributions of the source and target domains.Among them,the features are frequently utilized as a global object to perform inter-domain adaptation tasks,disregarding the coupling of task-related information(inter-domain invariant,intra-domain specific information) and task-irrelevant information(color contrast,image style) in the features.The situation makes it difficult for the model to comprehend the important information of features,resulting in sub-optimization.In consideration of the aforementioned issues,we propose an unsupervised domain adaptive classification method based on a task relevant feature separation network(TRFS),which helps the network extract the downstream task-related feature weight by learning the attention consistency between the features with inter-domain style mixed interference and the original features.Further,weight subtraction is used to obtain the task-irrelevant feature weight,then the task-related and irrelevant features are further pushed away by orthogonal function constraints to achieve feature separation.The task feature refinement separation layer is designed to reduce the confusion situation of alignment features and domain-specific features,as well as optimize the model’s classification and discrimination accuracy.Comprehensive experiment results demonstrate that the designed separation module has good plug-and-play performance,which can enhance the performance of other UDA methods.And the TRFS has obvious advantages over other advanced UDA methods,which achieves a classification accuracy of 73.6% in the Office-Home benchmark.

Key words: Feature decoupling, Task-relevant, Attention mechanism, Unsupervised domain adaptation, Image classification

中图分类号: 

  • TP391
[1] FAN C N,LIU P,XIAO T,et al.A Review of Deep Domain Adaptation:General Situation and Complex Situation[J].Acta Automatica Sinica,2021,47(3):515-548.
[2]LI J J,MENG L C,ZHANG K,et al.Review of Studies on Domain Adaptation[J].Computer Engineering,2021,47(6):1-13.
[3]GRETTON A,BORGWARDT K M,RASCH M J,et al.A kernel two-sample test[J].The Journal of Machine Learning Research,2012,13(1):723-773.
[4]LONG M,ZHU H,WANG J,et al.Deep transfer learning with joint adaptation networks[C]//International Conference on Machine Learning.PMLR,2017:2208-2217.
[5]LONG M,CAO Z,WANG J,et al.Conditional adversarial do-main adaptation[J].Advances in Neural Information Processing Systems,2018,31.
[6]TZENG E,HOFFMAN J,SAENKO K,et al.Adversarial dis-criminative domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:7167-7176.
[7]ZHU Y,ZHUANG F,WANG J,et al.Deep subdomain adaptation network for image classification[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(4):1713-1722.
[8]MA T.Generalization and equilibrium in generative adversarial nets(GANs)(invited talk)[C]//Proceedings of the 50th annual ACM SIGACT Symposium on Theory Cf computing.2018:2.
[9]NAM H,LEE H J,PARK J,et al.Reducing domain gap by reducing style bias[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:8690-8699.
[10]ANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adversarial training of neural networks[J].The Journal of Machine Learning Research,2016,17(1):2096-2030.
[11]BOUSMALIS K,TRIGEORGIS G,SILBERMAN N,et al.Domain separation networks[J].avXiv:1608.06019,2016.
[12]JIN X,LAN C,ZENG W,et al.Style normalization and restitution for domain generalization and adaptation[J].IEEE Transactions on Multimedia,2022,24:3636-3651.
[13]WEI G,LAN C,ZENG W,et al.ToAlign:Task-oriented Alignment for Unsupervised Domain Adaptation[J].Advances in Neural Information Processing Systems,2021,34:13834-13846.
[14]PAN S J,YANG Q.A survey on transfer learning[J].IEEE Transactions on Knowledge and Data Engineering,2009,22(10):1345-1359.
[15]DU Z,LI J,SU H,et al.Cross-domain gradient discrepancy minimization for unsupervised domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:3937-3946.
[16]LI J,DU Z,ZHU L,et al.Divergence-agnostic unsupervised domain adaptation by adversarial attacks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(11):8196-8211.
[17]WANG Z,HE X,YANG B,et al.Subdomain adaptation transfer learning network for fault diagnosis of roller bearings[J].IEEE Transactions on Industrial Electronics,2021,69(8):8430-8439.
[18]BEN-DAVID S,BLITZER J,CRAMMER K,et al.A theory of learning from different domains[J].Machine Learning,2010,79(1):151-175.
[19]YAN H,DING Y,LI P,et al.Mind the class weight bias:Weighted maximum mean discrepancy for unsupervised domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2272-2281.
[20]LUO Y W,REN C X.Conditional bures metric for domain adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:13989-13998.
[21]SUN B,SAENKO K.Deep coral:Correlation alignment for deep domain adaptation[C]//European Conference on Computer Vision.Springer,2016:443-450.
[22]ZELLINGER W,GRUBINGER T,LUGHOFER E,et al.Central moment discrepancy(cmd) for domain-invariant representation learning[J].arXiv:1702.08811,2017.
[23]ZHUANG F,CHENG X,LUO P,et al.Supervised representation learning:Transfer learning with deep autoencoders[C]//Twenty-Fourth International Joint Conference on Artificial Intelligence.2015.
[24]LI J,CHEN E,DING Z,et al.Maximum density divergence for domain adaptation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(11):3918-3930.
[25]LONG M,CAO Y,WANG J,et al.Learning transferable fea-tures with deep adaptation networks[C]//International Confe-rence on Machine Learning.PMLR,2015:97-105.
[26]CHEN H,LI L,CHEN J,et al.Unsupervised domain adaptation via double classifiers based on high confidence pseudo label[J].arXiv:2105.04729,2021.
[27]WANG J,CHEN Y,HAO S,et al.Balanced distribution adaptation for transfer learning[C]//2017 IEEE International Confe-rence on Data Mining(ICDM).IEEE,2017:1129-1134.
[28]PEI Z,CAO Z,LONG M,et al.Multi-adversarial domain adaptation[C]//Thirty-second AAAI Conference on Artificial Intelligence.2018:3934-3941.
[29]CREAGER E,MADRAS D,JACOBSEN J H,et al.Flexibly fair representation learning by disentanglement[C]//International Conference on Machine Learning.PMLR,2019:1436-1445.
[30]LEE H Y,TSENG H Y,HUANG J B,et al.Diverse image-to-image translation via disentangled representations[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:35-51.
[31]LIU S,ZHANG L,YANG X,et al.Unsupervised part segmentation through disentangling appearance and shape[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:8355-8364.
[32]LEE H Y,TSENG H Y,HUANG J B,et al.Diverse image-to-image translation via disentangled representations[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:35-51.
[33]SONG Z,KOYEJO O,ZHANG J.Toward a controllable disentanglement network[J].IEEE Transactions on Cybernetics,2022,52(4):2491-2504.
[34]CAI R,LI Z,WEI P,et al.Learning disentangled semantic representation for domain adaptation[C]//Proceedings of the Conference.NIH Public Access,IJCAI,2019:2060-2066.
[35]XU S,ZHANG J,LIU J.Image Style Transfer and Content-Style Disentanglement[J].arXiv:2111.15624,2021.
[36]GATYS L A,ECKER A S,BETHGE M.Image style transfer using convolutional neural networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:2414-2423.
[37]JOHNSON J,ALAHI A,FEI-FEI L.Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision.Cham:Springer,2016:694-711.
[38]HUANG X,BELONGIE S.Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:1501-1510.
[39]LV F,LIANG J,LI S,et al.Causality Inspired Representation Learning for Domain Generalization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:8046-8056.
[40]LI S,XIE B,LIN Q,et al.Generalized domain conditioned adaptation network[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(8):4093-4109.
[41]LIANG J,HU D,FENG J.Domain adaptation with auxiliary target domain-oriented classifier[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:16632-16642.
[42]BERTHELOT D,CARLINI N,GOODFELLOW I,et al.Mixmatch:A holistic approach to semi-supervised learning[J].arXiv:1905.02249,2019.
[43]HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[44]ZHU J Y,PARK T,ISOLA P,et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2223-2232.
[45]TANG H,CHEN K,JIA K.Unsupervised domain adaptationvia structurally regularized deep clustering[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:8725-8735.
[46]LIANG J,HU D,FENG J.Do we really need to access thesource data? source hypothesis transfer for unsupervised domain adaptation[C]//International Conference on Machine Learning.PMLR,2020:6028-6039.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发