计算机科学 ›› 2021, Vol. 48 ›› Issue (12): 278-285.doi: 10.11896/jsjkx.210900250

• 人工智能 • 上一篇    下一篇

检索式聊天机器人技术综述

吴俣1, 李舟军2   

  1. 1 微软亚洲研究院自然语言计算组 北京100080
    2 北京航空航天大学计算机学院 北京100191
  • 收稿日期:2020-03-20 修回日期:2020-12-20 出版日期:2021-12-15 发布日期:2021-11-26
  • 通讯作者: 李舟军(lizj@buaa.edu.cn)
  • 作者简介:yuwu1@microsoft.com
  • 基金资助:
    国家自然科学基金(U1636211,61672081);软件开发环境国家重点实验室课题(SKLSDE-2021ZX-18)

Survey on Retrieval-based Chatbots

WU Yu1, LI Zhou-jun2   

  1. 1 Natural Language Computing Group,Microsoft Research Asia,Beijing 100080,China
    2 School of Computer Science and Engineering,Beihang University,Beijing 100191,China
  • Received:2020-03-20 Revised:2020-12-20 Online:2021-12-15 Published:2021-11-26
  • About author:WU Yu,born in 1992,Ph.D,senior researcher.His main research interests include natural language processing and spoken language processing.
    LI Zhou-jun,born in 1963,Ph.D,professor,is a member of China Computer Federation.His main research interests include data mining,natural language processing,network and information security.
  • Supported by:
    National Natural Science Foundation of China(U1636211,61672081) and Fund of the State Key Laboratory of Software Development Environment(SKLSDE-2021ZX-18).

摘要: 随着自然语言处理技术的飞速发展以及互联网上对话语料的不断积累,闲聊导向对话系统(简称聊天机器人)取得了令人瞩目的进展,受到了学术界的广泛关注,并在产业界进行了初步的尝试。当前,聊天机器人分为检索式聊天机器人和生成式聊天机器人,而检索式聊天机器人由于其生成的回复流畅且计算资源消耗小,仍然是目前工业界聊天机器人的主要实现手段。文中首先简要介绍了检索式聊天机器人的研究背景、基本架构以及组成模块,重点阐述了回复选择模块的约束要求和相关数据集;然后,针对检索式聊天机器人中最为核心的回复选择技术,进行了深入分析与详细梳理。文中将近年来经典的回复选择技术归纳为如下4类:基于统计模型的方法、基于表示的神经网络模型的方法、基于交互的神经网络模型的方法以及基于预训练技术的方法,并指出了这4类方法的优点和不足。在此基础上,分析了目前检索式聊天机器人技术研究所面临的问题,并对其未来的发展趋势进行了展望。

关键词: 自然语言处理, 聊天机器人, 文本匹配, 回复选择, 预训练技术

Abstract: With the rapid progress of natural language processing techniques and the massive accessible conversational data on Internet,non-tasked oriented dialogue systems,also referred to as Chatbots,have achieved great success,and drawn attention from both academia and industry.Currently,there are two lines in chatbots research,retrieval-based chatbots and generation-based chatbots.Due to the fluent responses and low latency,retrieval-based chatbots is a common method in practice.This paperfirst briefly introduces the research background, basic structure and component modules of retrieval-based chatbots,and then illustrates the constraints of the response selection module and related data set in details.Subsequently,we summarize recent popular techniques for response selection problem,including:statistic method,representation-based neural network method,interaction-based neural network method,and pre-training-based method.Finally,we pose the challenges of chatbots and outline promising directions as future work.

Key words: Natural language processing, Chatbot, Text matching, Response selection, Pre-training technology

中图分类号: 

  • TP391
[1]WEIZENBAUM J.ELIZA:a computer program for the study of natural language communication between man and machine[J].Communications of the ACM,1966,9(1):36-45.
[2]RITTER A,CHERRY C,DOLAN W B.Data-driven response generation in social media[C]//Proceedings of the 2011 Confe-rence on Empirical Methods in Natural Language Processing.2011:583-593.
[3]JI Z,LU Z,LI H.An Information Retrieval Approach to Short Text Conversation[J].arXiv:1408.6988,2014.
[4]VINYALS O,LE Q.A neural conversational model[J].arXiv:1506.05869,2015.
[5]SUTSKEVER I,VINYALS O,LE Q V.Sequence to sequence learning with neural networks[C]//Advances in Neural Information Processing Systems.2014:3104-3112.
[6]ZHOU L,GAO J,LI D,et al.The design and implementation of xiaoice,an empathetic social chatbot.[J].Computational Linguistics,2020,46(1):53-93.
[7]LOWE R,POW N,SERBAN I,et al.The Ubuntu Dialogue Corpus:A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems[C]//Proceedings of the SIGDIAL 2015 Conference,The 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue.Prague,Czech Republic,2015:285-294.
[8]WANG H,LU Z,LI H,et al.A Dataset for Research on Short-Text Conversations[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.EMNLP,2013:935-945.
[9]WU Y,WU W,XING C,et al.Sequential Match Network:A New Architecture for Multi-turn Response Selection in Re-trieval-based Chatbots[C]//Proceedings of the 55th Annual Mee-ting of the Association for Computational Linguistics.2017:496-505.
[10]ZHANG Z,LI J,ZHU P,ZHAO H,LIU G.Modeling Multi-turn Conversation with Deep Utterance Aggregation[C]//Proceedings of the 27th International Conference on Computational Linguistics 2018:3740-3752.
[11]ZHANG S,DINAN E,URBANEK J,et al.Personalizing Dialogue Agents:I have a dog,do you have pets too?[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.ACL,2018:2204-2213.
[12]WELLECK S,WESTON J,SZLAM A,et al.Dialogue natural language inference [C]//Proceedings of the 57th Annual Mee-ting of the Association for Computational Linguistics.ACL,2019:3731-3741.
[13]BURGES C,SHAKED T,RENSHAW E,et al. Learning to rank using gradient descent[C]//Proceedings of the 22nd International Conference on Machine Learning (ICML-05):89-96.
[14]SPÄRCK J K.A Statistical Interpretation of Term Specificity and Its Application in Retrieval[J].Journal of Documentation,1972;28(1):11-21.
[15]BROWN P F,DELLA S A,DELLA P V J,et al.The mathema- tics of statistical machine translation:Parameter estimation [J].Computational Linguistics,1993;19(2):263-311.
[16]ZHAO X,JIANG J,WENG J,et al.Comparing twitter and traditional media using topic models[C]//European Conference on Information Retrieval.2011:338-349.
[17]WAGNER R A,FISCHER M J.The string-to-string correction problem[J].Journal of the ACM (JACM),1974,21(1):168-173.
[18]MACKAY D J.Information theory,inference and learning algorithms [M].Cambridge University Press,2003.
[19]BAEZA-YATES R,RIBEIRO-NETO B,et al.Modern information retrieval[M]//volume 463.ACM press,New York,1999.
[20]CHOI J,YOO K,LEE S.Learning to compose task-specific tree structures[C]//Thirty-Second AAAI Conference on Artificial Intelligence.2018:248-258.
[21]LIU X,KEVIN D,GAO J.Stochastic answer networks for natural language inference[J].arXiv:1804.07888,2018.
[22]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM.A Convolutional Neural Network for Modelling Sentences[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.2014:655-665.
[23]ZHOU X,DONG D,WU H,et al.Multi-view Response Selection for Human-Computer Conversation[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:372-381.
[24]YAN R,SONG Y,WU H.Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System[C]//Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval.2016:55-64.
[25]QIU X,HUANG X.Convolutional neural tensor network architecture for community-based question answering[C]//Procee-dings of the 24th International Conference on Artificial Intelligence.2015:1305-1311.
[26]WU Y,WU W,XING C,et al.A sequential matching framework for multi-turn response selection in retrieval-based chatbots [J].Computational Linguistics,2019,45(1),163-197.
[27]ZHOU X,LI L,DONG D,et al.Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2018:1118-1127.
[28]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[29]YIN W,SCHÜTZE H,XIANG B,et al.Abcnn:Attention-based convolutional neural network for modeling sentence pairs[J].Transactions of the Association for Computational Linguistics,2016(4):259-272.
[30]PANG L,LAN Y,GUO J,et al.Text matching as image recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2016:2793-2799.
[31]CHEN Q,ZHU X,LING Z H,et al.Enhanced LSTM for Natural Language Inference[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2017:1657-1668.
[32]SONG S,WANG C,PU X,et al.An Enhanced Convolutional Inference Model with Distillation for Retrieval-Based QA[C]//DASFAA.2021:511-515.
[33]PETERS M,NEUMANN M,IYYER M,et al.Deep contextua- lized word representations[C]//Proceedings of NAACL-HLT.2018:2227-2237.
[34]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[C]//Advances in Neural Information Processing Systems.2013:3111-3119.
[35]RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving language understanding by generative pre-training[OL].https://s3-us-west-2.amazonaws.com/openai-assets/researchcovers/languageunsupervised/language understanding paper.pdf,2018.
[36]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding [C]//Proceedings of NAACL-HLT.2019:4171-4186.
[37]LIU Y,OTT M,GOYAL N,et al.Roberta:A robustly opti- mized BERT pretraining approach[J].arXiv:1907.11692,2019.
[38]YANG Z,DAI Z,YANG Y,et al.Xlnet:Generalized autoregressive pretraining for language understanding[C]//Advances in Neural Information Processing Systems.2019:32-42.
[39]ZHANG Z,HAN X,LIU Z,et al.ERNIE:Enhanced Language Representation with Informative Entities[J].arXiv:1905.07129,2019.
[40]WHANG T,LEE D,LEE C,et al.Domain Adaptive Training BERT for Response Selection[J].arXiv:1908.04812.
[41]TALMOR A,HERZIG J,LOURIE N,et al.Commonsense QA:A Question Answering Challenge Targeting Commonsense Knowledge[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:4149-4158.
[42]DUA D,WANG Y,DASIGI P,et al.DROP:A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs[C]//Proceedings of North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:2368-2378.
[43]ZHOU K,ZHANG K,WU Y,et al.Unsupervised Context Rewriting for Open Domain Conversation[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.2017:1834-1844.
[44]YU J,QIU M,JIANG J,et al.Modelling domain relationships for transfer learning on retrieval-based question answering systems in e-commerce[C]//Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining.2018:682-690.
[1] 王胜, 张仰森, 陈若愚, 向尕. 基于细粒度差异特征的文本匹配方法[J]. 计算机科学, 2021, 48(8): 60-65.
[2] 王立梅, 朱旭光, 汪德嘉, 张勇, 邢春晓. 基于深度学习的民事案件判决结果分类方法研究[J]. 计算机科学, 2021, 48(8): 80-85.
[3] 裴莹, 李天祥, 王鏖清, 付加胜, 韩霄松. 基于新闻的国际天然气价格趋势预测方法[J]. 计算机科学, 2021, 48(6A): 235-239.
[4] 吕乐宾, 刘群, 彭露, 邓维斌, 王崇宇. 结合多粒度信息的文本匹配融合模型[J]. 计算机科学, 2021, 48(6): 196-201.
[5] 仝鑫, 王斌君, 王润正, 潘孝勤. 面向自然语言处理的深度学习对抗样本综述[J]. 计算机科学, 2021, 48(1): 258-267.
[6] 陆龙龙, 陈统, 潘敏学, 张天. CodeSearcher:基于自然语言功能描述的代码查询[J]. 计算机科学, 2020, 47(9): 1-9.
[7] 田野, 寿黎但, 陈珂, 骆歆远, 陈刚. 基于字段嵌入的数据库自然语言查询接口[J]. 计算机科学, 2020, 47(9): 60-66.
[8] 张迎, 张宜飞, 王中卿, 王红玲. 基于主次关系特征的自动文摘方法[J]. 计算机科学, 2020, 47(6A): 6-11.
[9] 张浩洋, 周良. 改进的GHSOM算法在民航航空法规知识地图构建中的应用[J]. 计算机科学, 2020, 47(6A): 429-435.
[10] 吴小坤, 赵甜芳. 自然语言处理技术在社会传播学中的应用研究和前景展望[J]. 计算机科学, 2020, 47(6): 184-193.
[11] 胡超文, 杨亚连, 邬昌兴. 基于深度学习的隐式篇章关系识别综述[J]. 计算机科学, 2020, 47(4): 157-163.
[12] 余珊珊, 苏锦钿, 李鹏飞. 一种基于自注意力的句子情感分类方法[J]. 计算机科学, 2020, 47(4): 204-210.
[13] 李舟军,范宇,吴贤杰. 面向自然语言处理的预训练技术研究综述[J]. 计算机科学, 2020, 47(3): 162-173.
[14] 苗益, 赵增顺, 杨雨露, 徐宁, 杨皓然, 孙骞. 图像描述技术综述[J]. 计算机科学, 2020, 47(12): 149-160.
[15] 霍丹, 张生杰, 万路军. 基于上下文的情感词向量混合模型[J]. 计算机科学, 2020, 47(11A): 28-34.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 崔琼,李建华,王宏,南明莉. 基于节点修复的网络化指挥信息系统弹性分析模型[J]. 计算机科学, 2018, 45(4): 117 -121 .
[2] 王振朝,侯欢欢,连蕊. 抑制CMT中乱序程度的路径优化方案[J]. 计算机科学, 2018, 45(4): 122 -125 .
[3] 施超,谢在鹏,柳晗,吕鑫. 基于稳定匹配的容器部署策略的优化[J]. 计算机科学, 2018, 45(4): 131 -136 .
[4] 郭帅,刘亮,秦小麟. 用户偏好约束的空间关键词范围查询处理方法[J]. 计算机科学, 2018, 45(4): 182 -189 .
[5] 文俊浩,孙光辉,李顺. 基于用户聚类和移动上下文的矩阵分解推荐算法研究[J]. 计算机科学, 2018, 45(4): 215 -219 .
[6] 贾伟,华庆一,张敏军,陈锐,姬翔,王博. 基于改进粒子群优化的移动界面模式聚类算法[J]. 计算机科学, 2018, 45(4): 220 -226 .
[7] 魏芹双,武优西,刘靖宇,朱怀忠. 基于密度约束和间隙约束的对比模式挖掘[J]. 计算机科学, 2018, 45(4): 252 -256 .
[8] 童泽平,李涛,李立杰,任亮. 基于随机需求与产能限制的供应链协同优化研究[J]. 计算机科学, 2018, 45(4): 260 -265 .
[9] 蔡莉,梁宇,朱扬勇,何婧. 数据质量的历史沿革和发展趋势[J]. 计算机科学, 2018, 45(4): 1 -10 .
[10] 徐周波,张鵾,宁黎华,古天龙. 图编辑距离概述[J]. 计算机科学, 2018, 45(4): 11 -18 .