计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (8): 142-149.doi: 10.11896/jsjkx.220800040

• 人工智能 • 上一篇    下一篇

融合知识的领域自适应方法综述

崔福伟1, 吴璇璇2, 陈钰枫2, 刘健2, 徐金安2   

  1. 1 北京交通大学电子信息工程学院 北京 100044
    2 北京交通大学计算机与信息技术学院 北京 100044
  • 收稿日期:2022-08-03 修回日期:2022-11-10 出版日期:2023-08-15 发布日期:2023-08-02
  • 通讯作者: 徐金安(jaxu@bjtu.edu.cn)
  • 作者简介:(fuweicui@bjtu.edu.cn)
  • 基金资助:
    国家重点研发计划(2019YFB1405200);国家自然科学基金(61976015,61976016,61876198,61370130)

Survey of Domain Adaptive Methods with Knowledge Integrating

CUI Fuwei1, WU Xuanxuan2, CHEN Yufeng2, LIU Jian2, XU Jin'an2   

  1. 1 School of Electronic Information Engineering,Beijing Jiaotong University,Beijing 100044,China
    2 School of Computer and Information Technology,Beijing Jiaotong University,Beijing 100044,China
  • Received:2022-08-03 Revised:2022-11-10 Online:2023-08-15 Published:2023-08-02
  • About author:CUI Fuwei,born in 1990,Ph.D,is a student member of China Computer Federation.His main research interests include natural language processing,text generation,dialog system,etc.
    XU Jin'an,born in 1970,Ph.D,professor,Ph.D supervisor,is a senior member of China Computer Federation.His main research interests include natural language processing,machine translation,know-ledge graph and its application,text emotion analysis,automatic summarization,question answering,dialogue system,human-machine interaction,etc.
  • Supported by:
    National Key R & D Program of China(2019YFB1405200) and National Natural Science Foundation of China(61976015,61976016,61876198,61370130).

PDF (PC)

1516

摘要： 训练基于数据驱动的模型时,常假设源域和目标域的数据分布相同,但是,在实际场景中,这一假设通常不成立,因此容易造成模型的泛化能力较差的问题。为提高模型的泛化能力,领域自适应方法应运而生,其通过学习源域和目标域的数据特征来对齐两域数据分布,使得在源域数据上训练好的模型在有少量数据标签或者没有数据标签的目标域上也具有较好表现。为了进一步提高模型的泛化能力,现有研究探索将知识融入领域自适应方法中,该技术具有较高的实用价值。文中首先概述了融合知识的领域自适应方法的发展背景和相关综述的研究现状;其次对领域自适应的问题定义和理论基础进行了介绍;然后给出了一种融合知识的领域自适应方法的分类体系,并对其中的一些代表性方法进行了概述;最后,通过对该领域挑战性问题的分析,预测了融合知识的领域自适应方法未来的研究方向,以期为相关的研究提供一定的参考。

关键词: 泛化能力, 领域自适应, 融合知识, 分类体系

Abstract: When training a data-driven model,it is often assumed that the data distribution of the source domain and the target domain are the same.However,in the natural scenario,this assumption is usually not tenable,and it is easy to cause poor generalization ability of the model.Domain adaptation is a method proposed to improve the generalization ability of the model.It aligns the data distribution of the source domain and the target domain by learning the data characteristics of the two domains,so that the model trained in the source domain data can also perform well in the target domain with a small number of data labels or without data labels.In order to further improve the generalization ability of the model,existing researches have explored the know-ledge integrating into domain adaptive methods,which has high practical value.Firstly,we summarizes the development background of domain adaptive methods with knowledge integrating and the research status of related reviews.Then,the problem defi-nition and theoretical basis of domain adaptation are introduced.After that,a classification system of domain adaptive methods with knowledge integrating is presented,and some representative methods are summarized.Finally,through the analysis of the challenging problems in this field,the future research directions of domain adaptive methods with knowledge integrating are predicted,in the hope of providing some reference for related research.

Key words: Generalization ability, Domain adaptation, Knowledge integrating, Classification system

中图分类号: 

  • TP391
[1]PAN S J,QIANG Y.A Survey on Transfer Learning[J].IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359.
[2]JING J.A Literature Survey on Domain Adaptation of Statistical Classifiers[J].British Journal of Psychiatry,2008,131:83-89.
[3]CHEN S T.Research on domain adaptive algorithm[D].Guang-zhou:South China University of Technology,2020.
[4]XU M.Research on domain adaptive learning algorithm and its application [D].Wuxi:Jiangnan University,2014.
[5]LIU J W,SUN Z K,LUO X L.Progress in domain adaptivelearning [J].Acta Automatica Sinica,2014,(8):1576-1600.
[6]LI J J,MENG L C,ZHANG K,et al.Overview of domain adaptation research [J].Computer Engineering,2021,47(6):13.
[7]FAN M,CAI Z,ZHANG T,et al.A survey of deep domainadaptation based on label set classification[J].Multimedia Tools and Applications,2022,81:39545-39576.
[8]RAMPONI A,PLANK B.Neural Unsupervised Domain Adaptation in NLP---A Survey[J].arXiv:2006.00632,2020.
[9]SAUNDERS D.Domain adaptation and multi-domain adaptation for neural machine translation:A survey[J].arXiv:2104.06951,2021.
[10]DAVIS J,DOMINGOS P.Deep Transfer via Second-OrderMarkov Logic[C]//International Conference on Machine Lear-ning.ACM,2009.
[11]SHEN Y,LEONARD D,PAVEL P S,et al.Word-based Domain Adaptation for Neural Machine Translation[C]//Proceedings of the 15th International Conference on Spoken Language Translation.2018:31-38.
[12]PRATIK J,MAYANK M,BAMDEV M.Geometry-aware domain adaptation for unsupervised alignment of word embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:3052-3058.
[13]GEORGE-EDUARD Z,RĂZVAN-ALEXANDRU S.DomainAdaptation in Multilingual and Multi-Domain Monolingual Settings for Complex Word Identification[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2022:70-80.
[14]MIKEL A,GORKA L,CHAKAVEH S,et al.Adding syntactic structure to bilingual terminology for improved domain adaptation[C]//Proceedings of the 2nd Deep Machine Translation Workshop.2016:39-46.
[15]ZHUANG C,TIEYUN Q.Bridge-Based Active Domain Adaptation for Aspect Term Extraction[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing.2021:317-327.
[16]KAI Z,QI L,ZHENYA H,et al.Graph Adaptive SemanticTransfer for Cross-domain Sentiment Classification[J].arXiv:2205.08772,2022.
[17]LI T,CHEN X,DONG Z,et al.Domain-Adaptive Text Classification with Structured Knowledge from Unlabeled Data[J].arXiv:2206.09591,2022.
[18]SUCHIN G,ANA M,SWABHA S,et al.Don't Stop Pretrai-ning:Adapt Language Models to Domains and Tasks[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:8342-8360.
[19]ROEE A,YOAV G.Unsupervised domain clusters in pretrained language models[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:7747-7763.
[20]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[J].arXiv:1406.2661,2014.
[21]EYAL B,NADAV O,ROI R.PADA:Example-based PromptLearning for on-the-fly Adaptation to Unseen Domains[J].Transactions of the Association for Computational Linguistics,2022,10:414-433.
[22]FAN B,ALAN R,WEI X.Pre-train or Annotate? Domain Adaptation with a Constrained Budget[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.2021:5002-5015.
[23]HU X,LIU B,SHU L,et al.DomBERT:Domain-oriented language model for aspect-based sentiment analysis[C]//Findings of the Association for Computational Linguistics:EMNLP 2020.2020:1725-1731.
[24]EYHARABIDE V,BEKKOUCH I,CONSTANTIN N D.KnowledgeGraph Embedding-Based Domain Adaptation for Musical Instrument Recognition[J].Computers,2021,10(8):94.
[25]ZHANG J W,ZHU J Q,YANG Y,et al.Knowledge-Enhanced Domain Adaptation in Few-Shot Relation Classification[C]//Proceedings of the 27th ACM SIGKDD Conference on Know-ledge Discovery & Data Mining(KDD '21).2021.
[26]DEEPANWAY G,DEVAMANYU H,ABHINABA R,et al.KinGDOM:Knowledge-Guided DOMain Adaptation for Sentiment Analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:3198-3210.
[27]KIM Y,RUSH A M.Sequence-Level Knowledge Distillation[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1317-1327.
[28]GORDON M,KEVIN D.Distill,Adapt,Distill:Training Small,In-Domain Models for Neural Machine Translation[C]//Proceedings of the Fourth Workshop on Neural Generation and Translation.2020.
[29]CHOI D H,HONGSEOK C,HYUNJU L.Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2022.
[30]KRISTEN H,JIAN W,AKSHAY H,et al.Domain-specificknowledge distillation yields smaller and better models for conversational commerce[C]//Proceedings of The Fifth Workshop on e-Commerce and NLP(ECNLP 5).2022.
[31]YAO Y Z,HUANG S H, WANG W H,et al.Adapt-and-Distill:Developing Small,Fast and Effective Pretrained Language Models for Domains[C]//Findings of the Association for Computational Linguistics:ACL-IJCNLP 2021.2021.
[32]PAN H J,WANG C,QIU M,et al.Meta-KD:A Meta Know-ledge Distillation Framework for Language Model Compression across Domains[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 1:Long Papers).2021.
[33]CURREY A,PRASHANT M,GEORGIANA D.Distilling multiple domains for neural machine translation[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).2020.
[34]HU M,WU Y,ZHAO S,et al.Domain-Invariant Feature Distillation for Cross-Domain Sentiment Classification[C]//Procee-dings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).2019:5559-5568.
[35]HINTON G,VINYALS O,DEAN J.Distilling the knowledge ina neural network[C]//NIPS 2014 Deep Learning Workshop.2014.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发