计算机科学 ›› 2021, Vol. 48 ›› Issue (5): 232-238.doi: 10.11896/jsjkx.200600092

所属专题: 自然语言处理 虚拟专题

• 人工智能 • 上一篇    下一篇

面向任务的基于深度学习的多轮对话系统与技术

姚冬1, 李舟军2, 陈舒玮2, 季震1, 张锐1, 宋磊1, 蓝海波1   

  1. 1 国网冀北电力有限公司 北京100053
    2 北京航空航天大学计算机学院 北京100191
  • 收稿日期:2020-06-16 修回日期:2020-07-17 出版日期:2021-05-15 发布日期:2021-05-09
  • 通讯作者: 李舟军(lizj@buaa.edu.cn)
  • 基金资助:
    国家自然科学基金(U1636211,61672081);软件开发环境国家重点实验室课题(SKLSDE-2019ZX-17);国网人工智能技术在调控运行全过程安全管控中的应用研究项目(520101180044)

Task-oriented Dialogue System and Technology Based on Deep Learning

YAO Dong1, LI Zhou-jun 2, CHEN Shu-wei2, JI Zhen1, ZHANG Rui1, SONG Lei1, LAN Hai-bo1   

  1. 1 State Grid Jibei Electric Company Limited,Beijing 100053,China
    2 School of Computer Science and Engineering,Beihang University,Beijing 100191,China
  • Received:2020-06-16 Revised:2020-07-17 Online:2021-05-15 Published:2021-05-09
  • About author:YAO Dong,born in 1985,postgraduate.His main research interests include power system dispatching and control operation,application of artificial intelligence in power system dispatching and control operation.(yao.dong@jibei.sgcc.com.cn)
    LI Zhou-jun,born in 1963,Ph.D,professor,is a member of China Computer Federation.His main research interests include data mining,natural language processing,network and information security.
  • Supported by:
    National Natural Science Foundation of China(U1636211,61672081),Fund of the State Key Laboraty of Software Development Environment(SKLSDE-2019ZX-17) and Research on the Application of Artificial Intelligence in the Safety Control of the Whole Process of Regulation and Operation in State Grid(520101180044).

摘要: 自然语言是人类智慧的结晶,以自然语言的形式与计算机进行交互是人们长久以来的期待。随着自然语言处理技术的发展与深度学习方法的兴起,人机对话系统成为了新的研究热点。人机对话系统按照功能可以分为任务导向型对话系统、闲聊型对话系统、问答型对话系统。任务导向型对话系统是一种典型的人机对话系统,旨在帮助用户完成某些特定的任务,有着十分重要的学术意义和应用价值。文中系统地阐述了一种在实际工程应用中的任务导向型对话系统的通用框架,主要包括自然语言理解、对话管理以及自然语言生成3个部分;介绍了上述各部分所采用的经典深度学习和机器学习方法。最后,对自然语言理解任务进行了实证性的实验验证与分析,结果表明文中内容可以为任务导向型对话系统的构建提供有效指导。

关键词: 对话管理, 任务导向型对话系统, 深度学习, 自然语言理解, 自然语言生成

Abstract: Natural language is the crystallization of human wisdom,and interacting with computers in the form of natural language has long been expected.With the development of natural language processing technology and the rise of deep learning methods,human-computer dialogue systems have become a new research hotspot.Human-computer dialogue systems can be divided into task-oriented dialogue systems,chit-chat-oriented dialogue systems,and question-and-answer dialogue systems accor-ding to their functions.The task-oriented dialogue system is a typical man-machine dialogue system,which aims to help users complete certain specific tasks,and has very important academic significance and application value.This paper systematically illustrates the general framework of task-oriented dialogue systems in practical engineering applications,including natural language understanding,dialogue management,and natural language generation.Then,the classical deep learning and machine learning methods used in the above parts are introduced.Finally,the task of natural language understanding is empirically verified and analyzed.This paper can provide effective guidance for the construction of a task-oriented dialogue system.

Key words: Deep learning, Dialog management, Natural language generation, Natural language understanding, Task-oriented dialogue system

中图分类号: 

  • TP391
[1]ZHAO Y.Research on the Key Technologies of Customizable Knowledge-Based Chatbots [D].Beijing:Beihang University,2018.
[2]CHEN H,LIU X,YIN D,et al.A survey on dialogue systems:Recent advances and new frontiers[J].ACM SIGKDD Explorations Newsletter,2017,19(2):25-35.
[3]YOUNG S J.Probabilistic methods in spoken-dialogue systems[J].Philosophical Transactions of the Royal Society of London A:Mathematical,Physical and Engineering Sciences,2000,358(1769):1389-1402.
[4]HASHEMI H B,ASIAEE A,KRAFT R.Query intent detection using convolutional neural networks[C]//InternationalConfe-rence on Web Search and Data Mining,Workshop on Query Understanding.2016.
[5]SHEN Y,HE X,GAO J,et al.Learning semantic representations using convolutional neural networks for web search[C]//Proceedings of the 23rd International Conference on World Wide Web.ACM,2014:373-374.
[6]DEORAS A,SARIKAYA R.Deep belief network based semantic taggers for spoken language understanding[C]//Interspeech.2013:2713-2717.
[7]MESNIL G,HE X,DENG L,et al.Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding[C]//Interspeech.2013:3771-3775.
[8]YAO K,ZWEIG G,HWANG M Y,et al.Recurrent neural net-works for language understanding[C]//Interspeech.2013:2524-2528.
[9]HOCHREITER S,SCHMIDHUBER J.Long Short-TermMemory[J].Neural Computation,1997,9(8):1735-1780.
[10]HAKKANI-TÜR D,TÜR G,CELIKYILMAZ A,et al.Multi-Domain Joint Semantic Frame Parsing Using Bi-Directional RNN-LSTM[C]//Interspeech.2016:715-719.
[11]GOO C W,GAO G,HSU Y K,et al.Slot-Gated Modeling for Joint Slot Filling and Intent Prediction[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2018:753-757.
[12]LEVIN E,PIERACCINI R,ECKERTW.A stochastic model of human-machine interaction for learning dialog strategies[J].IEEE Transactions on Speech and Audio Processing,2000,8(1):11-23.
[13]SINGH S,LITMAN D,KEARNSM,et al.Optimizing dialogue management with reinforcement learning:Experiments with the NJFun system[J].Journal of Artificial Intelligence Research,2002,16:105-133.
[14]YOUNG S,GAŠIĆ M,KEIZER S,et al.The hidden information state model:A practical framework for POMDP-based spoken dialogue management[J].Computer Speech & Language,2010,24(2):150-174.
[15]WALKER M A,RAMBOW O C,ROGATI M.Training a sentence planner for spoken dialogue using boosting[J].Computer Speech & Language,2002,16(3/4):409-433.
[16]STENT A,PRASAD R,WALKER M.Trainable sentence planning for complex information presentation in spoken dialog systems[C]//Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics.Association for Computational Linguistics,2004:79.
[17]WEN T H,GASIC M,KIM D,et al.Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking[J].arXiv:1508.01755,2015.
[18]ZHOU H,HUANG M.Context-aware natural language generation for spoken dialogue systems[C]//Proceedings of the 26th International Conference on Computational Linguistics:Technical Papers.2016:2032-2041.
[19]ZHOU P,SHI W,TIAN J,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.2016:207-212.
[20]KIM Y.Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).2014:1746-1751.
[21]HUANG Z,XU W,YU K.Bidirectional LSTM-CRF models for sequence tagging[J].arXiv:1508.01991,2015.
[22]LAFFERTY J D,MCCALLUM A,PEREIRA F C N.Condi-tional Random Fields:Probabilistic Models for Segmenting and Labeling Sequence Data[J].Proceedings of Icml,2001,3(2):282-289.
[23]KALCHBRENNER N,BLUNSOM P.Recurrent continuoustranslation models[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.2013:1700-1709.
[24]LUONG T,PHAM H,MANNING C D.Effective Approaches to Attention-based Neural Machine Translation[C]//Procee-dings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:1412-1421.
[25]ZHU S,YU K.Encoder-decoder with focus-mechanism for se-quence labelling based apoken language understanding[C]//2017 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).IEEE,2017:5675-5679.
[26]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:4171-4186.
[27]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[28]LIU B,LANE I.Attention-Based Recurrent Neural NetworkModels for Joint Intent Detection and Slot Filling[C]//Interspeech 2016.2016.
[29]ZHANG C W,LI Y L,DU N,et al.Joint Slot Filling and Intent Detection via Capsule Neural Networks[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:5259-5267.
[30]HOCHREITER S,SCHMIDHUBE R,JÜRGE N.Long Short-Term Memory[J].Neural Computation,1997,9(8):1735-1780.
[31]CHO K,VAN MERRIËNBOER B,GULCEHRE C,et al.Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).2014:1724-1734.
[32]SU P H,VANDYKE D,GASIC M,et al.Learning from Real Users:Rating Dialogue Success with Nerual Networks for Reinforcement Learning in Spoken Dialogue Systems[OL].https://www.oalib.com/paper/4050158#.YFwoVPliras.
[33]MRKŠIĆ N,ÓŚEAGHDHA D,WEN T H,et al.Neural belief tracker:Data-driven dialogue state tracking[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada,Association for Computational Linguistics,2017:1777-1788.
[34]Y O,CHEN M,DAI X Y,et al.Dialogue State Tracking with Explicit Slot Connection Modeling[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:34-40.
[35]LI X,CHEN Y N,LI L,et al.End-to-End Task-CompletionNeural Dialogue Systems[J].arXiv:1703.01008.
[36]SU P H,BUDZIANOWSKI P,ULTES S,et al.Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management[J].arXiv:1707.00130.
[37]WEN T H,GASIC M,KIM D,et al.Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking[C]//Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue.Prague,Czech Republic,Association for Computational Linguistics,2015:275-284.
[38]ZHOU H,HUANG M,ZHU X.Context-aware natural lan-guage generation for spoken dialogue systems[C]//COLING.2016:2032-2041.
[39]LI Y M,YAO K S,QINL B,et al.Slot-consistent NLG forTask-oriented Dialogue Systems with Iterative Rectification Network[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:97-106.
[40]ZHAO X,CAO Y.Overview of the NLPCC 2018 Shared Task:Spoken Language Understanding in Task-Oriented Dialog Systems[OL].http://tcci.ccf.org.cn/conference/2018/papers/EV33.pdf.
[41]SONG Y,SHI S,LI J,et al.Directional Skip-Gram:ExplicitlyDistinguishing Left and Right Context for Word Embedding[OL].https://www.researchgate.net/publication/325447242_Directional_Skip-Gram_Explicitly_Distinguishing_Left_and_Right_Context_for_Word_Embeddings.2018.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[9] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[10] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[11] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[12] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[13] 王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰.
基于多源迁移学习的大坝裂缝检测
Dam Crack Detection Based on Multi-source Transfer Learning
计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[14] 楚玉春, 龚航, 王学芳, 刘培顺.
基于YOLOv4的目标检测知识蒸馏算法研究
Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4
计算机科学, 2022, 49(6A): 337-344. https://doi.org/10.11896/jsjkx.210600204
[15] 祝文韬, 兰先超, 罗唤霖, 岳彬, 汪洋.
改进Faster R-CNN的光学遥感飞机目标检测
Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN
计算机科学, 2022, 49(6A): 378-383. https://doi.org/10.11896/jsjkx.210300121
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!