计算机科学 ›› 2021, Vol. 48 ›› Issue (8): 60-65.doi: 10.11896/jsjkx.200700008
王胜, 张仰森, 陈若愚, 向尕
WANG Sheng, ZHANG Yang-sen, CHEN Ruo-yu, XIANG Ga
摘要： 文本匹配是检索系统中的关键技术之一。针对现有文本匹配模型对文本语义差异捕获不准确的问题,文中提出了一种基于细粒度差异特征的文本匹配方法。首先,使用预训练模型作为基础模型对匹配文本进行语义的抽取与初步匹配;然后,引入对抗学习的思想,在模型的编码阶段人为构造虚拟对抗样本进行训练,以提升模型的学习能力与泛化能力;最后,通过引入文本的细粒度差异特征,纠正文本匹配的初步预测结果,有效提升了模型对细粒度差异特征的捕获能力,进而提升了文本匹配模型的性能。在两个数据集上进行了实验验证,其中在LCQMC数据集上的实验结果显示,所提方法在ACC性能指标上达到了88.96%,优于已知的最好模型。
中图分类号:
| [1]LI X.Research on paragraph retrieval technology for questionanswering system[D].Chengdu:University of Science and Technology of China,2010. [2]SALTON G,BUCKLEY C.Term-weighting approaches in automatic text retrieval[J].Information Processing & Management,1988,24(5):513-523. [3]SONG F,CROFT W B.A general language model for information retrieval[C]//Proceedings of the Eighth International Conference on Information and Knowledge Management.1999:316-321. [4]LE Q,MIKOLOV T.Distributed representations of sentencesand documents[C]//International Conference on Machine Learning.2014:1188-1196. [5]LOGESWARAN L,LEE H.An efficient framework for learning sentence representations[J].arXiv:1803.02893,2018. [6]CER D,YANG Y,KONG S,et al.Universal sentence encoder[J].arXiv:1803.11175,2018. [7]YIN W,SCHÜTZE H,XIANG B,et al.Abcnn:Attention-based convolutional neural network for modeling sentence pairs[J].Transactions of the Association for Computational Linguistics,2016,4:259-272. [8]CHEN Q,ZHU X,LING Z,et al.Enhanced lstm for natural language inference[J].arXiv:1609.06038,2016. [9]WANG Z,HAMZA W,FLORIAN R.Bilateral multi-perspective matching for natural language sentences[J]. arXiv:1702.03814,2017. [10]RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving language understanding with unsupervised learning[R/OL].Technical Report,OpenAI,2018.https://openai.com/blog/language-unsupervised/. [11]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training ofdeep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018. [12]RADFORD A,WU J,CHILD R,et al.Language models are unsupervised multitask learners[J].OpenAI Blog,2019,1(8):9. [13]SUN Y,WANG S,LI Y,et al.Ernie:Enhanced representation through knowledge integration[J].arXiv:1904.09223,2019. [14]RAFFEL C,SHAZEER N,ROBERTS A,et al.Exploring the limits of transfer learning with a unified text-to-text transformer[J].arXiv:1910.10683,2019. [15]HU W,DANG A,TAN Y.A Survey of State-of-the-Art Short Text Matching Algorithms[C]//International Conference on Data Mining and Big Data.Singapore:Springer,2019:211-219. [16]SAKATA W,SHIBATA T,TANAKA R,et al.FAQ retrieval using query-question similarity and BERT-based query-answer relevance[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.2019:1113-1116. [17]WU Y,WANG R J.Application of BERT-Based Semantic Ma-tching Algorithm in Question Answering System[J].Instrumentation Technology,2020(6):19-22,30. [18]WANG N Z.Research on improved text representation model based on BERT[D].Southwest University.2019. [19]GOODFELLOW I J,SHLENS J,SZEGEDY C.Explaining and harnessing adversarial examples[J].arXiv:1412.6572,2014. [20]ZHU C,CHENG Y,GAN Z,et al.Freelb:Enhanced adversarial training for natural language understanding[C]//International Conference on Learning Representations.2019. [21]YAN J,BRACEWELL D B,REN F,et al.A semantic analyzer for aiding emotion recognition in Chinese[C]//International Conference on Intelligent Computing.Berlin,Heidelberg:Springer,2006:893-901. [22]SONG Y,SHI S,LI J,et al.Directional skip-gram:Explicitlydistinguishing left and right context for word embeddings[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 2 (Short Papers).2018:175-180. [23]LIU X,CHEN Q,DENG C,et al.Lcqmc:A large-scale chinese question matching corpus[C]//Proceedings of the 27th International Conference on Computational Linguistics.2018:1952-1962. [24]ZHANG X,LU W,ZHANG G,et al.Chinese Sentence Semantic Matching Based on Multi-Granularity Fusion Model[C]//Paci-fic-Asia Conference on Knowledge Discovery and Data Mining.Cham:Springer,2020:246-257. [25]LIU W,ZHOU P,ZHAO Z,et al.K-BERT:Enabling Language Representation with Knowledge Graph[C]//AAAI.2020:2901-2908. [26]CHEN J,CAO C,JIANG X.SiBert:Enhanced Chinese Pre-trained Language Model with Sentence Insertion[C]//Procee-dings of The 12th Language Resources and Evaluation Confe-rence.2020:2405-2412. [27]MENG Y,WU W,WANG F,et al.Glyce:Glyph-vectors forChinese character representations[C]//Advances in Neural Information Processing Systems.2019:2746-2757. [28]CUI Y,CHE W,LIU T,et al.Pre-training with whole word masking for chinese bert[J].arXiv:1906.08101,2019. [29]KOEHN P.Statistical significance tests for machine translation evaluation[C]//Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing.2004:388-395. |
| [1] | 吕晓锋, 赵书良, 高恒达, 武永亮, 张宝奇. 基于异质信息网的短文本特征扩充方法 Short Texts Feautre Enrichment Method Based on Heterogeneous Information Network 计算机科学, 2022, 49(9): 92-100. https://doi.org/10.11896/jsjkx.210700241 |
| [2] | 曹晓雯, 梁美玉, 鲁康康. 基于细粒度语义推理的跨媒体双路对抗哈希学习模型 Fine-grained Semantic Reasoning Based Cross-media Dual-way Adversarial Hashing Learning Model 计算机科学, 2022, 49(9): 123-131. https://doi.org/10.11896/jsjkx.220600011 |
| [3] | 曾志贤, 曹建军, 翁年凤, 蒋国权, 徐滨. 基于注意力机制的细粒度语义关联视频-文本跨模态实体分辨 Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism 计算机科学, 2022, 49(7): 106-112. https://doi.org/10.11896/jsjkx.210500224 |
| [4] | 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018 |
| [5] | 赵丹丹, 黄德根, 孟佳娜, 董宇, 张攀. 基于BERT-GRU-ATT模型的中文实体关系分类 Chinese Entity Relations Classification Based on BERT-GRU-ATT 计算机科学, 2022, 49(6): 319-325. https://doi.org/10.11896/jsjkx.210600123 |
| [6] | 刘硕, 王庚润, 彭建华, 李柯. 基于混合字词特征的中文短文本分类算法 Chinese Short Text Classification Algorithm Based on Hybrid Features of Characters and Words 计算机科学, 2022, 49(4): 282-287. https://doi.org/10.11896/jsjkx.210200027 |
| [7] | 侯宏旭, 孙硕, 乌尼尔. 蒙汉神经机器翻译研究综述 Survey of Mongolian-Chinese Neural Machine Translation 计算机科学, 2022, 49(1): 31-40. https://doi.org/10.11896/jsjkx.210900006 |
| [8] | 刘立波, 苟婷婷. 融合深度典型相关分析和对抗学习的跨模态检索 Cross-modal Retrieval Combining Deep Canonical Correlation Analysis and Adversarial Learning 计算机科学, 2021, 48(9): 200-207. https://doi.org/10.11896/jsjkx.200600119 |
| [9] | 汤世征, 张岩峰. DragDL:一种易用的深度学习模型可视化构建系统 DragDL:An Easy-to-Use Graphical DL Model Construction System 计算机科学, 2021, 48(8): 220-225. https://doi.org/10.11896/jsjkx.200900045 |
| [10] | 吕乐宾, 刘群, 彭露, 邓维斌, 王崇宇. 结合多粒度信息的文本匹配融合模型 Text Matching Fusion Model Combining Multi-granularity Information 计算机科学, 2021, 48(6): 196-201. https://doi.org/10.11896/jsjkx.200700100 |
| [11] | 詹瑞, 雷印杰, 陈训敏, 叶书函. 基于多重差异特征网络的街景变化检测 Street Scene Change Detection Based on Multiple Difference Features Network 计算机科学, 2021, 48(2): 142-147. https://doi.org/10.11896/jsjkx.200500158 |
| [12] | 吴俣, 李舟军. 检索式聊天机器人技术综述 Survey on Retrieval-based Chatbots 计算机科学, 2021, 48(12): 278-285. https://doi.org/10.11896/jsjkx.210900250 |
| [13] | 黄欣, 雷刚, 曹远龙, 陆明名. 基于深度学习的交互式问答研究综述 Review on Interactive Question Answering Techniques Based on Deep Learning 计算机科学, 2021, 48(12): 286-296. https://doi.org/10.11896/jsjkx.210100209 |
| [14] | 张玉帅, 赵欢, 李博. 基于BERT和BiLSTM的语义槽填充 Semantic Slot Filling Based on BERT and BiLSTM 计算机科学, 2021, 48(1): 247-252. https://doi.org/10.11896/jsjkx.191200088 |
| [15] | 倪海清, 刘丹, 史梦雨. 基于语义感知的中文短文本摘要生成模型 Chinese Short Text Summarization Generation Model Based on Semantic-aware 计算机科学, 2020, 47(6): 74-78. https://doi.org/10.11896/jsjkx.190600006 |
|
||
首页