Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (5): 232-238.doi: 10.11896/jsjkx.200600092

Special Issue: Natural Language Processing

• Artificial Intelligence • Previous Articles     Next Articles

Task-oriented Dialogue System and Technology Based on Deep Learning

YAO Dong1, LI Zhou-jun 2, CHEN Shu-wei2, JI Zhen1, ZHANG Rui1, SONG Lei1, LAN Hai-bo1   

  1. 1 State Grid Jibei Electric Company Limited,Beijing 100053,China
    2 School of Computer Science and Engineering,Beihang University,Beijing 100191,China
  • Received:2020-06-16 Revised:2020-07-17 Online:2021-05-15 Published:2021-05-09
  • About author:YAO Dong,born in 1985,postgraduate.His main research interests include power system dispatching and control operation,application of artificial intelligence in power system dispatching and control operation.(yao.dong@jibei.sgcc.com.cn)
    LI Zhou-jun,born in 1963,Ph.D,professor,is a member of China Computer Federation.His main research interests include data mining,natural language processing,network and information security.
  • Supported by:
    National Natural Science Foundation of China(U1636211,61672081),Fund of the State Key Laboraty of Software Development Environment(SKLSDE-2019ZX-17) and Research on the Application of Artificial Intelligence in the Safety Control of the Whole Process of Regulation and Operation in State Grid(520101180044).

PDF (PC)

2792

Abstract: Natural language is the crystallization of human wisdom,and interacting with computers in the form of natural language has long been expected.With the development of natural language processing technology and the rise of deep learning methods,human-computer dialogue systems have become a new research hotspot.Human-computer dialogue systems can be divided into task-oriented dialogue systems,chit-chat-oriented dialogue systems,and question-and-answer dialogue systems accor-ding to their functions.The task-oriented dialogue system is a typical man-machine dialogue system,which aims to help users complete certain specific tasks,and has very important academic significance and application value.This paper systematically illustrates the general framework of task-oriented dialogue systems in practical engineering applications,including natural language understanding,dialogue management,and natural language generation.Then,the classical deep learning and machine learning methods used in the above parts are introduced.Finally,the task of natural language understanding is empirically verified and analyzed.This paper can provide effective guidance for the construction of a task-oriented dialogue system.

Key words: Deep learning, Dialog management, Natural language generation, Natural language understanding, Task-oriented dialogue system

CLC Number: 

  • TP391
[1]ZHAO Y.Research on the Key Technologies of Customizable Knowledge-Based Chatbots [D].Beijing:Beihang University,2018.
[2]CHEN H,LIU X,YIN D,et al.A survey on dialogue systems:Recent advances and new frontiers[J].ACM SIGKDD Explorations Newsletter,2017,19(2):25-35.
[3]YOUNG S J.Probabilistic methods in spoken-dialogue systems[J].Philosophical Transactions of the Royal Society of London A:Mathematical,Physical and Engineering Sciences,2000,358(1769):1389-1402.
[4]HASHEMI H B,ASIAEE A,KRAFT R.Query intent detection using convolutional neural networks[C]//InternationalConfe-rence on Web Search and Data Mining,Workshop on Query Understanding.2016.
[5]SHEN Y,HE X,GAO J,et al.Learning semantic representations using convolutional neural networks for web search[C]//Proceedings of the 23rd International Conference on World Wide Web.ACM,2014:373-374.
[6]DEORAS A,SARIKAYA R.Deep belief network based semantic taggers for spoken language understanding[C]//Interspeech.2013:2713-2717.
[7]MESNIL G,HE X,DENG L,et al.Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding[C]//Interspeech.2013:3771-3775.
[8]YAO K,ZWEIG G,HWANG M Y,et al.Recurrent neural net-works for language understanding[C]//Interspeech.2013:2524-2528.
[9]HOCHREITER S,SCHMIDHUBER J.Long Short-TermMemory[J].Neural Computation,1997,9(8):1735-1780.
[10]HAKKANI-TÜR D,TÜR G,CELIKYILMAZ A,et al.Multi-Domain Joint Semantic Frame Parsing Using Bi-Directional RNN-LSTM[C]//Interspeech.2016:715-719.
[11]GOO C W,GAO G,HSU Y K,et al.Slot-Gated Modeling for Joint Slot Filling and Intent Prediction[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2018:753-757.
[12]LEVIN E,PIERACCINI R,ECKERTW.A stochastic model of human-machine interaction for learning dialog strategies[J].IEEE Transactions on Speech and Audio Processing,2000,8(1):11-23.
[13]SINGH S,LITMAN D,KEARNSM,et al.Optimizing dialogue management with reinforcement learning:Experiments with the NJFun system[J].Journal of Artificial Intelligence Research,2002,16:105-133.
[14]YOUNG S,GAŠIĆ M,KEIZER S,et al.The hidden information state model:A practical framework for POMDP-based spoken dialogue management[J].Computer Speech & Language,2010,24(2):150-174.
[15]WALKER M A,RAMBOW O C,ROGATI M.Training a sentence planner for spoken dialogue using boosting[J].Computer Speech & Language,2002,16(3/4):409-433.
[16]STENT A,PRASAD R,WALKER M.Trainable sentence planning for complex information presentation in spoken dialog systems[C]//Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics.Association for Computational Linguistics,2004:79.
[17]WEN T H,GASIC M,KIM D,et al.Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking[J].arXiv:1508.01755,2015.
[18]ZHOU H,HUANG M.Context-aware natural language generation for spoken dialogue systems[C]//Proceedings of the 26th International Conference on Computational Linguistics:Technical Papers.2016:2032-2041.
[19]ZHOU P,SHI W,TIAN J,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.2016:207-212.
[20]KIM Y.Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).2014:1746-1751.
[21]HUANG Z,XU W,YU K.Bidirectional LSTM-CRF models for sequence tagging[J].arXiv:1508.01991,2015.
[22]LAFFERTY J D,MCCALLUM A,PEREIRA F C N.Condi-tional Random Fields:Probabilistic Models for Segmenting and Labeling Sequence Data[J].Proceedings of Icml,2001,3(2):282-289.
[23]KALCHBRENNER N,BLUNSOM P.Recurrent continuoustranslation models[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.2013:1700-1709.
[24]LUONG T,PHAM H,MANNING C D.Effective Approaches to Attention-based Neural Machine Translation[C]//Procee-dings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:1412-1421.
[25]ZHU S,YU K.Encoder-decoder with focus-mechanism for se-quence labelling based apoken language understanding[C]//2017 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).IEEE,2017:5675-5679.
[26]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:4171-4186.
[27]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[28]LIU B,LANE I.Attention-Based Recurrent Neural NetworkModels for Joint Intent Detection and Slot Filling[C]//Interspeech 2016.2016.
[29]ZHANG C W,LI Y L,DU N,et al.Joint Slot Filling and Intent Detection via Capsule Neural Networks[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:5259-5267.
[30]HOCHREITER S,SCHMIDHUBE R,JÜRGE N.Long Short-Term Memory[J].Neural Computation,1997,9(8):1735-1780.
[31]CHO K,VAN MERRIËNBOER B,GULCEHRE C,et al.Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).2014:1724-1734.
[32]SU P H,VANDYKE D,GASIC M,et al.Learning from Real Users:Rating Dialogue Success with Nerual Networks for Reinforcement Learning in Spoken Dialogue Systems[OL].https://www.oalib.com/paper/4050158#.YFwoVPliras.
[33]MRKŠIĆ N,ÓŚEAGHDHA D,WEN T H,et al.Neural belief tracker:Data-driven dialogue state tracking[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada,Association for Computational Linguistics,2017:1777-1788.
[34]Y O,CHEN M,DAI X Y,et al.Dialogue State Tracking with Explicit Slot Connection Modeling[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:34-40.
[35]LI X,CHEN Y N,LI L,et al.End-to-End Task-CompletionNeural Dialogue Systems[J].arXiv:1703.01008.
[36]SU P H,BUDZIANOWSKI P,ULTES S,et al.Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management[J].arXiv:1707.00130.
[37]WEN T H,GASIC M,KIM D,et al.Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking[C]//Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue.Prague,Czech Republic,Association for Computational Linguistics,2015:275-284.
[38]ZHOU H,HUANG M,ZHU X.Context-aware natural lan-guage generation for spoken dialogue systems[C]//COLING.2016:2032-2041.
[39]LI Y M,YAO K S,QINL B,et al.Slot-consistent NLG forTask-oriented Dialogue Systems with Iterative Rectification Network[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:97-106.
[40]ZHAO X,CAO Y.Overview of the NLPCC 2018 Shared Task:Spoken Language Understanding in Task-Oriented Dialog Systems[OL].http://tcci.ccf.org.cn/conference/2018/papers/EV33.pdf.
[41]SONG Y,SHI S,LI J,et al.Directional Skip-Gram:ExplicitlyDistinguishing Left and Right Context for Word Embedding[OL].https://www.researchgate.net/publication/325447242_Directional_Skip-Gram_Explicitly_Distinguishing_Left_and_Right_Context_for_Word_Embeddings.2018.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[9] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[10] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[11] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[12] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[13] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
[14] SUN Fu-quan, CUI Zhi-qing, ZOU Peng, ZHANG Kun. Brain Tumor Segmentation Algorithm Based on Multi-scale Features [J]. Computer Science, 2022, 49(6A): 12-16.
[15] KANG Yan, XU Yu-long, KOU Yong-qi, XIE Si-yu, YANG Xue-kun, LI Hao. Drug-Drug Interaction Prediction Based on Transformer and LSTM [J]. Computer Science, 2022, 49(6A): 17-21.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.