计算机科学 ›› 2021, Vol. 48 ›› Issue (6): 196-201.doi: 10.11896/jsjkx.200700100

• 人工智能 • 上一篇    下一篇

结合多粒度信息的文本匹配融合模型

吕乐宾, 刘群, 彭露, 邓维斌, 王崇宇   

  1. 重庆邮电大学计算智能重庆市重点实验室 重庆400065
  • 收稿日期:2020-07-15 修回日期:2020-08-20 出版日期:2021-06-15 发布日期:2021-06-03
  • 通讯作者: 刘群(liuqun@cqupt.edu.cn)
  • 基金资助:
    国家重点研发计划资助项目(2018YFC0832100, 2018YFC0832102);国家自然科学重点基金项目(61936001)

Text Matching Fusion Model Combining Multi-granularity Information

LYU Le-bin, LIU Qun, PENG Lu, DENG Wei-bin , WANG Chong-yu   

  1. Chongqing Key Laboratory of Computational Intelligence,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
  • Received:2020-07-15 Revised:2020-08-20 Online:2021-06-15 Published:2021-06-03
  • About author:LYU Le-bin,born in 1994,master.His main research interests include natural language processing and so on.(lebinlv@foxmail.com)
    LIU Qun,born in 1969,Ph.D,profes-sor,is a member of China Computer Federation.Her main research interests include data mining,complex network and so on.
  • Supported by:
    National Key Research and Development Program of China(2018YFC0832100, 2018YFC0832102) and Key Program of National Natural Science Foundation of China(61936001).

摘要: 常规的文本匹配模型大致分为基于表示的文本匹配模型和基于交互的文本匹配模型。由于基于表示的文本匹配模型容易失去语义焦点,而基于交互的文本匹配模型会忽视全局信息,文中提出了结合多粒度信息的文本匹配融合模型。该模型通过交互注意力和表示注意力将两种文本匹配模型进行了融合,然后利用卷积神经网络提取了文本中存在的多个不同级别的粒度信息,使得模型既能抓住局部的重要信息又能获取全局的语义信息。在3组不同的文本匹配任务上的实验结果表明,所提出的模型在NDCG@5评价指标上分别优于其他最优模型5.3%,0.4%,1.5%。通过提取文本中的多个粒度信息并结合交互注意力和表示注意力,提出的模型能够有效地关注不同级别的文本信息,解决了传统模型在文本匹配过程中易失去语义焦点和忽视全局信息的问题。

关键词: 表示注意力, 多粒度信息, 交互注意力, 粒度网络, 文本匹配

Abstract: Conventional text matching methods are basically divided into representational text matching models and interaction-based text matching models.Since the representation-based text matching model is easy to lose semantic focus and the interaction-based text matching model ignores global information,a text matching fusion model combining multi-granularity information is proposed in this paper.This model fuses two text matching models through interactive attention and expressing attention,and then uses convolutional neural networks to extract multiple different levels of granularity information presented in the text.Then the local important information and global semantic information can be captured.The experimental results on three different text matching tasks show that the proposed model outperform other optimal models by 5.3%,0.4%,1.5% on the NDCG@5 evaluation index respectively.By extracting multiple granularity information of the text and combining interactive attention and expressed attention,the proposed model can effectively pay attention to the text information of different levels,and solve the problem of losing semantics and ignoring global information during the text matching process in the traditional models.

Key words: Expressive attention, Granular network, Interactive attention, Multi-granularity information, Text matching

中图分类号: 

  • TP391
[1]HUANG P S,HE X,GAO J,et al.Learning deep structured semantic models for web search using click through data[C]//Proceedings of the 22nd ACM international conference on Conference on in-formation & knowledge management.California:ACM Press,2013:2333-2338.
[2]PANG L,LAN Y,GUO J,et al.Text matching as image recognition[C]//Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.Arizona:AAAI Press,2016:2793-2799.
[3]JÄRVELIN K,KEKÄLÄINEN J.Cumulated gain-based evaluation of IR techniques[J].ACM Transactions on Information Systems (TOIS),2002,20(4),422-446.
[4] SHEN Y,HE X,GAO J,et al.Learning semantic representations using convolutional neural networks for web search[C]//International Conference on World Wide Web.Seoul:ACM Press,2014:373-374.
[5]WAN S,LAN Y,GUO J,et al.A deep architecture for semantic matching with multiple positional sentence representations[C]//Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.Arizona:AAAI Press,2016:2835-2841.
[6]GONG Y,LUO H,ZHANG J.Natural Language Inference over Interaction Space.International Conference on Learning Representations [EB/OL].[2020-7-11].https://arxiv.org/pdf/1709.04348.pdf.
[7] HU B,LU Z,LI H,et al.Convolutional Neural Network Architectures for Matching Natural Language Sentences[C]//Advances in Neural Information Processing Systems.Montreal:MIT Press,2015:2042-2050.
[8]XIONG C,DAI Z,CALLAN J,et al.End-to-End Neural Ad-hoc Ranking with Kernel Pooling[J].In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval.Tokyo:ACM Press,2017,51(cd):55-64.
[9]DAI Z,XIONG C,CALLAN J,et al.Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search[C]//Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining.California:ACM Press,2018:126-134.
[10]PENNINGTON J,SOCHER R,MANNING C.Glove:GlobalVectors for Word Representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing.Doha,Qatar:ACL Press,2014:1532-1543.
[11]SEO M,KEMBHAVI A,FARHADI A,et al.Bidirectional Attention Flow for Machine Comprehension.arXiv:Computation and Language [EB/OL].[2020-7-11].https://arxiv.org/pdf/1611.01603.pdf.
[12]XIE S,GIRSHICK R,DOLLAR P,et al.Aggregated ResidualTransformations for Deep Neural Networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Hawaii:IEEE Press,2017:5987-5995.
[13]YANG Y,YIH W,MEEK C.WikiQA:A Challenge Dataset for Open-Domain Question Answering[C]//In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal:ACL Press,2015:2013-2018.
[14]BOWMAN S,ANGELI G,POTTS C,et al.A large annotated corpus for learning natural language inference[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal:ACL Press,2015:632-642.
[15]NAKOV P,MÁRQUEZ L,MOSCHITTI A,et al.SemEval-2016 Task 3:Community Question Answering.ArXiv:Information Retrieval[EB/OL].[2020-07-11].http://arxiv.org/abs/1912.00730.pdf.
[16]FAN Y,PANG L,HOU J,et al.MatchZoo:A Toolkit for Deep Text Matching.ArXiv:Information Retrieval[EB/OL].[2020-07-11].https://arxiv.org/pdf/1707.07270v1.pdf.
[17]MITRA B,DIAZ F,CRASWELL N.Learning to match using local and distributed representations of text for web search[C]//Proceedings of the 26th International Conference on World Wide Web.Australia:ACM Press,2017:1291-1299.
[18]GUO J,FAN Y,AI Q,et al.A Deep Relevance Matching Mo-del for Ad-hoc Retrieval[C]//Conference on Information and Knowledge Management.Indianapolis:ACM Press,2016:55-64.
[19]YANG Z, LAN Q, GUO J,et al.A deeptop-k relevance ma-tching model for ad-hoc retrieval[C]//China Conference on Information Retrieval.Guilin:Springer Press,2018:16-27.
[1] 吕晓锋, 赵书良, 高恒达, 武永亮, 张宝奇.
基于异质信息网的短文本特征扩充方法
Short Texts Feautre Enrichment Method Based on Heterogeneous Information Network
计算机科学, 2022, 49(9): 92-100. https://doi.org/10.11896/jsjkx.210700241
[2] 潘志豪, 曾碧, 廖文雄, 魏鹏飞, 文松.
基于交互注意力图卷积网络的方面情感分类
Interactive Attention Graph Convolutional Networks for Aspect-based Sentiment Classification
计算机科学, 2022, 49(3): 294-300. https://doi.org/10.11896/jsjkx.210100180
[3] 王胜, 张仰森, 陈若愚, 向尕.
基于细粒度差异特征的文本匹配方法
Text Matching Method Based on Fine-grained Difference Features
计算机科学, 2021, 48(8): 60-65. https://doi.org/10.11896/jsjkx.200700008
[4] 陈卓, 王国胤, 刘群.
结合多粒度特征融合的自然场景文本检测方法
Natural Scene Text Detection Algorithm Combining Multi-granularity Feature Fusion
计算机科学, 2021, 48(12): 243-248. https://doi.org/10.11896/jsjkx.201000154
[5] 吴俣, 李舟军.
检索式聊天机器人技术综述
Survey on Retrieval-based Chatbots
计算机科学, 2021, 48(12): 278-285. https://doi.org/10.11896/jsjkx.210900250
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!