Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (8): 142-149.doi: 10.11896/jsjkx.220800040

• Artificial Intelligence • Previous Articles     Next Articles

Survey of Domain Adaptive Methods with Knowledge Integrating

CUI Fuwei1, WU Xuanxuan2, CHEN Yufeng2, LIU Jian2, XU Jin'an2   

  1. 1 School of Electronic Information Engineering,Beijing Jiaotong University,Beijing 100044,China
    2 School of Computer and Information Technology,Beijing Jiaotong University,Beijing 100044,China
  • Received:2022-08-03 Revised:2022-11-10 Online:2023-08-15 Published:2023-08-02
  • About author:CUI Fuwei,born in 1990,Ph.D,is a student member of China Computer Federation.His main research interests include natural language processing,text generation,dialog system,etc.
    XU Jin'an,born in 1970,Ph.D,professor,Ph.D supervisor,is a senior member of China Computer Federation.His main research interests include natural language processing,machine translation,know-ledge graph and its application,text emotion analysis,automatic summarization,question answering,dialogue system,human-machine interaction,etc.
  • Supported by:
    National Key R & D Program of China(2019YFB1405200) and National Natural Science Foundation of China(61976015,61976016,61876198,61370130).

PDF (PC)

389

Abstract: When training a data-driven model,it is often assumed that the data distribution of the source domain and the target domain are the same.However,in the natural scenario,this assumption is usually not tenable,and it is easy to cause poor generalization ability of the model.Domain adaptation is a method proposed to improve the generalization ability of the model.It aligns the data distribution of the source domain and the target domain by learning the data characteristics of the two domains,so that the model trained in the source domain data can also perform well in the target domain with a small number of data labels or without data labels.In order to further improve the generalization ability of the model,existing researches have explored the know-ledge integrating into domain adaptive methods,which has high practical value.Firstly,we summarizes the development background of domain adaptive methods with knowledge integrating and the research status of related reviews.Then,the problem defi-nition and theoretical basis of domain adaptation are introduced.After that,a classification system of domain adaptive methods with knowledge integrating is presented,and some representative methods are summarized.Finally,through the analysis of the challenging problems in this field,the future research directions of domain adaptive methods with knowledge integrating are predicted,in the hope of providing some reference for related research.

Key words: Generalization ability, Domain adaptation, Knowledge integrating, Classification system

CLC Number: 

  • TP391
[1]PAN S J,QIANG Y.A Survey on Transfer Learning[J].IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359.
[2]JING J.A Literature Survey on Domain Adaptation of Statistical Classifiers[J].British Journal of Psychiatry,2008,131:83-89.
[3]CHEN S T.Research on domain adaptive algorithm[D].Guang-zhou:South China University of Technology,2020.
[4]XU M.Research on domain adaptive learning algorithm and its application [D].Wuxi:Jiangnan University,2014.
[5]LIU J W,SUN Z K,LUO X L.Progress in domain adaptivelearning [J].Acta Automatica Sinica,2014,(8):1576-1600.
[6]LI J J,MENG L C,ZHANG K,et al.Overview of domain adaptation research [J].Computer Engineering,2021,47(6):13.
[7]FAN M,CAI Z,ZHANG T,et al.A survey of deep domainadaptation based on label set classification[J].Multimedia Tools and Applications,2022,81:39545-39576.
[8]RAMPONI A,PLANK B.Neural Unsupervised Domain Adaptation in NLP---A Survey[J].arXiv:2006.00632,2020.
[9]SAUNDERS D.Domain adaptation and multi-domain adaptation for neural machine translation:A survey[J].arXiv:2104.06951,2021.
[10]DAVIS J,DOMINGOS P.Deep Transfer via Second-OrderMarkov Logic[C]//International Conference on Machine Lear-ning.ACM,2009.
[11]SHEN Y,LEONARD D,PAVEL P S,et al.Word-based Domain Adaptation for Neural Machine Translation[C]//Proceedings of the 15th International Conference on Spoken Language Translation.2018:31-38.
[12]PRATIK J,MAYANK M,BAMDEV M.Geometry-aware domain adaptation for unsupervised alignment of word embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:3052-3058.
[13]GEORGE-EDUARD Z,RĂZVAN-ALEXANDRU S.DomainAdaptation in Multilingual and Multi-Domain Monolingual Settings for Complex Word Identification[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2022:70-80.
[14]MIKEL A,GORKA L,CHAKAVEH S,et al.Adding syntactic structure to bilingual terminology for improved domain adaptation[C]//Proceedings of the 2nd Deep Machine Translation Workshop.2016:39-46.
[15]ZHUANG C,TIEYUN Q.Bridge-Based Active Domain Adaptation for Aspect Term Extraction[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing.2021:317-327.
[16]KAI Z,QI L,ZHENYA H,et al.Graph Adaptive SemanticTransfer for Cross-domain Sentiment Classification[J].arXiv:2205.08772,2022.
[17]LI T,CHEN X,DONG Z,et al.Domain-Adaptive Text Classification with Structured Knowledge from Unlabeled Data[J].arXiv:2206.09591,2022.
[18]SUCHIN G,ANA M,SWABHA S,et al.Don't Stop Pretrai-ning:Adapt Language Models to Domains and Tasks[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:8342-8360.
[19]ROEE A,YOAV G.Unsupervised domain clusters in pretrained language models[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:7747-7763.
[20]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[J].arXiv:1406.2661,2014.
[21]EYAL B,NADAV O,ROI R.PADA:Example-based PromptLearning for on-the-fly Adaptation to Unseen Domains[J].Transactions of the Association for Computational Linguistics,2022,10:414-433.
[22]FAN B,ALAN R,WEI X.Pre-train or Annotate? Domain Adaptation with a Constrained Budget[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.2021:5002-5015.
[23]HU X,LIU B,SHU L,et al.DomBERT:Domain-oriented language model for aspect-based sentiment analysis[C]//Findings of the Association for Computational Linguistics:EMNLP 2020.2020:1725-1731.
[24]EYHARABIDE V,BEKKOUCH I,CONSTANTIN N D.KnowledgeGraph Embedding-Based Domain Adaptation for Musical Instrument Recognition[J].Computers,2021,10(8):94.
[25]ZHANG J W,ZHU J Q,YANG Y,et al.Knowledge-Enhanced Domain Adaptation in Few-Shot Relation Classification[C]//Proceedings of the 27th ACM SIGKDD Conference on Know-ledge Discovery & Data Mining(KDD '21).2021.
[26]DEEPANWAY G,DEVAMANYU H,ABHINABA R,et al.KinGDOM:Knowledge-Guided DOMain Adaptation for Sentiment Analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:3198-3210.
[27]KIM Y,RUSH A M.Sequence-Level Knowledge Distillation[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1317-1327.
[28]GORDON M,KEVIN D.Distill,Adapt,Distill:Training Small,In-Domain Models for Neural Machine Translation[C]//Proceedings of the Fourth Workshop on Neural Generation and Translation.2020.
[29]CHOI D H,HONGSEOK C,HYUNJU L.Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2022.
[30]KRISTEN H,JIAN W,AKSHAY H,et al.Domain-specificknowledge distillation yields smaller and better models for conversational commerce[C]//Proceedings of The Fifth Workshop on e-Commerce and NLP(ECNLP 5).2022.
[31]YAO Y Z,HUANG S H, WANG W H,et al.Adapt-and-Distill:Developing Small,Fast and Effective Pretrained Language Models for Domains[C]//Findings of the Association for Computational Linguistics:ACL-IJCNLP 2021.2021.
[32]PAN H J,WANG C,QIU M,et al.Meta-KD:A Meta Know-ledge Distillation Framework for Language Model Compression across Domains[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 1:Long Papers).2021.
[33]CURREY A,PRASHANT M,GEORGIANA D.Distilling multiple domains for neural machine translation[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).2020.
[34]HU M,WU Y,ZHAO S,et al.Domain-Invariant Feature Distillation for Cross-Domain Sentiment Classification[C]//Procee-dings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).2019:5559-5568.
[35]HINTON G,VINYALS O,DEAN J.Distilling the knowledge ina neural network[C]//NIPS 2014 Deep Learning Workshop.2014.
[1] TIAN Tian-yi, SUN Fu-ming. Tumor Recognition Method Based on Domain Adaptive Algorithm [J]. Computer Science, 2022, 49(12): 250-256.
[2] JIN Yu-jie, CHU Xu, WANG Ya-sha, ZHAO Jun-feng. Variational Domain Adaptation Driven Semantic Segmentation of Urban Scenes [J]. Computer Science, 2022, 49(11): 126-133.
[3] NING Qiu-yi, SHI Xiao-jing, DUAN Xiang-yu, ZHANG Min. Unsupervised Domain Adaptation Based on Style Aware [J]. Computer Science, 2022, 49(1): 271-278.
[4] LIU Kai, ZHANG Hong-jun, CHEN Fei-qiong. Name Entity Recognition for Military Based on Domain Adaptive Embedding [J]. Computer Science, 2022, 49(1): 292-297.
[5] WU Lan, WANG Han, LI Bin-quan. Unsupervised Domain Adaptive Method Based on Optimal Selection of Self-supervised Tasks [J]. Computer Science, 2021, 48(6A): 357-363.
[6] MA Chuang, TIAN Qing, SUN He-yang, CAO Meng, MA Ting-huai. Unsupervised Domain Adaptation Based on Weighting Dual Biases [J]. Computer Science, 2021, 48(2): 217-223.
[7] YUAN Chen-hui, CHENG Chun-ling. Deep Domain Adaptation Algorithm Based on PE Divergence Instance Filtering [J]. Computer Science, 2020, 47(8): 151-156.
[8] YANG Pei-jian, WU Xiao-fu, ZHANG Suo-fei, ZHOU Quan. Semantic Segmentation Transfer Algorithm Based on Atrous Convolution Discriminator [J]. Computer Science, 2020, 47(11): 174-178.
[9] XU Shu-yan, HAN Li-xin, XU Guo-xia. Domain Adaptation Algorithm Based on Tensor Decomposition [J]. Computer Science, 2019, 46(12): 89-94.
[10] TAN Xun-tao, GU Yi-yi, RUAN Tong, YUAN Yu-bo. Confidence Interval Method for Classification Usability Evaluation of Data Sets [J]. Computer Science, 2019, 46(1): 78-85.
[11] MENG Xiao-long YANG Yan WANG Hong-jun XIAO Wen-chao. Occasion Determination of Clustering Ensemble [J]. Computer Science, 2015, 42(7): 48-51.
[12] MENG Juan, HU Gu-yu, PAN Zhi-song and ZHOU Yu-huan. Research and Perspective on Domain Adaptation Learning Algorithms [J]. Computer Science, 2015, 42(10): 7-12.
[13] GU Xin and WANG Shi-tong. Novel Domain Transfer Learning Approach Using Minimum Enclosing Ball [J]. Computer Science, 2013, 40(7): 187-191.
[14] YU Xu,YANG Jing,XIE Zhi-qiang. Research on Virtual Sample Generation Technology [J]. Computer Science, 2011, 38(3): 16-19.
[15] . [J]. Computer Science, 2008, 35(3): 158-160.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.