基于上下文信息的口语意图检测方法

doi:10.11896/jsjkx.181202269

摘要/Abstract

摘要： 近年来,随着人工智能的发展与智能设备的普及,人机智能对话技术得到了广泛的关注。口语语义理解是口语对话系统中的一项重要任务,而口语意图检测是口语语义理解中的关键环节。由于多轮对话中存在语义缺失、框架表示以及意图转换等复杂的语言现象,因此面向多轮对话的意图检测任务十分具有挑战性。为了解决上述难题,文中提出了基于门控机制的信息共享网络,充分利用了多轮对话中的上下文信息来提升检测性能。具体而言,首先结合字音特征构建当前轮文本和上下文文本的初始表示,以减小语音识别错误对语义表示的影响;其次,使用基于层级化注意力机制的语义编码器得到当前轮和上下文文本的深层语义表示,包含由字到句再到多轮文本的多级语义信息;最后,通过在多任务学习框架中引入门控机制来构建基于门控机制的信息共享网络,使用上下文语义信息辅助当前轮文本的意图检测。实验结果表明,所提方法能够高效地利用上下文信息来提升口语意图检测效果,在全国知识图谱与语义计算大会(CCKS2018)技术评测任务2的数据集上达到了88.1%的准确率(Acc值)和88.0%的综合正确率(F1值),相比于已有的方法显著提升了性能。

关键词: 口语语义理解, 门控神经网络, 上下文信息, 意图检测

Abstract: In recent years,with the development of artificial intelligence and the popularization of smart devices,human-computer intelligent dialogue technology has received extensive attention.Spoken language understanding is an important task dialogue system,and spoken language intention detection is a key technology in spoken language understanding.Due to complex language phenomena such as semantic missing,frame representation and intent conversion in multiple rounds of dialogue,the intent detection task for spoken language is very challenging.In order to solve the above problems,a gated mechanism based information sharing neural network method was proposed in this paper,which can take advantages of contextual information in multiple rounds of dialogue to improve detection performance.Specifically,first the current round text and context text initial representation are constructed in combination with the phonetic features to reduce the impact of speech recognition errors on semantic representation.Secondly,a semantic encoder based on hierarchical attention mechanism is used to obtain deep semantic representations of the current round and contextual text,including multi-level semantic information from word to sentence to multiple rounds of text.Finally,the gated mechaniam based information sharing neural network is constructed to use the context semantic information to help the intent detection of the current round of text.The experimental results show that the proposed method can effectively use context information to improve the detection of spoken language intentions,and achieves 88.1% accuracy and 88.0% F1 value in dataset of CCKS2018 shared task-2,which is significantly improved performance compared with the existing methods.

Key words: Context information, Gated neural network, Intent detection, Spoken language understanding

中图分类号:

TP391

徐扬,王建成,刘启元,李寿山. 基于上下文信息的口语意图检测方法[J]. 计算机科学, 2020, 47(1): 205-211. https://doi.org/10.11896/jsjkx.181202269

XU Yang,WANG Jian-cheng,LIU Qi-yuan,LI Shou-shan. Intention Detection in Spoken Language Based on Context Information[J]. Computer Science, 2020, 47(1): 205-211. https://doi.org/10.11896/jsjkx.181202269

参考文献

[1]WANG Y,REN F J,QUAN C Q.A Summary of Research on Dialogue Management Methods in Spoken Dialogue System[J].Computer Science,2015,42(6):1-7,27.
[2]CHEN H,LIU X,YIN D,et al.A survey on dialogue systems:Recent advances and new frontiers[J].ACM SIGKDD Explorations Newsletter,2017,19(2):25-35.
[3]HENDERSON M,THOMSON B,WILLIAMS J D.The second dialog state tracking challenge[C]∥Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse andDia-logue (SIGDIAL).ACL.Stroudsburg,PA.2014:263-272.
[4]ZONG C Q,WU H,HUANG T Y,et al.Analysis of Spoken Dia- log Corpus in Restricted Domain[C]∥Proceedings of the 5th National Conference on Computational Languages.1999:115-122.
[5]SONG H Y,ZHANG W N,LIU T.DQN based Policy Learning for Open Domain Multi-turn Dialogues[J].Journal of Chinese Information Processing,2018,32(7):99-108,136.
[6]SENEFF S.TINA:A natural language system for spoken language applications[J].Computational Iinguistics,1992,18(1):61-86.
[7]YAN P,ZHENG F,XU M.Robust parsing in spoken dialogue systems[C]∥Seventh European Conference on Speech Communication and Technology.Academic.Amsterdam.2001.
[8]HUANG Y F,ZHENG F,YAN P J,et al.The Design and Implementation of Campus Navigation System:Easy Nav[J].Journal of Chinese Information Processing,2001,15(4):36-41.
[9]DENG Y,XU B,HUANG T.Chinese spoken language understanding across domain[C]∥Sixth International Conference on Spoken Language Processing.IEEE,2000.
[10]MINKER W,BENNACEF S K,GAUVAIN J L.A stochastic case frame approach for natural language understanding[C]∥Fourth International Conference on Spoken Language Proces-sing.IEEE,1996.
[11]HAFFNER P,TUR G,WRIGHT J H.Optimizing SVMs for complex call classification[C]∥2003 IEEE International Conference on Acoustics,Speech,and Signal Processing(ICASSP’03).IEEE,2003.
[12]FREUND Y,SCHAPIRE R E.A decision-theoretic generalization of on-line learning and an application to boosting[J].Journal of computer and system sciences,1997,55(1):119-139.
[13]SARIKAYA R,HINTON G E,RAMABHADRAN B.Deep belief nets for natural language call-routing [C]∥Proceeding of the IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2011:5680-5683.
[14]TUR G,DENG L,HAKKANI-TÜR D,et al.Towards deeper understanding:Deep convex networks for semantic utterance classification [C]∥Proceeding of the IEEE International Confe-rence on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2012:5045-5048.
[15]HASHEMI H B,ASIAEE A,KRAFT R.Query intent detection using convolutional neural networks [C]∥International Confe-rence on Web Search and Data Mining,Workshop on Query Understanding.New York:ACM,2016.
[16]RAVURI S,STOICKE A.A comparative study of neural network models for lexical intent classification[C]∥2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).IEEE,2015:368-374.
[17]LIU B,LANE I.Attention-based recurrent neural network mo- dels for joint intent detection and slot filling[J].arXiv:1609.01454.
[18]FIRDAUS M,BHATNAGAR S,EKBAL A,et al.Intent Detection for Spoken Language Understanding Using a Deep Ensemble Model[C]∥Pacific Rim International Conference on Artificial Intelligence.Cham:Springer,2018:629-642.
[19]BARAHONA L M R,GASIC M,MRKŠIC＇ N,et al.Exploiting Sentence and Context Representations in Deep Neural Models for Spoken Language Understanding[J].arXiv:1610.04120,2016.
[20]XIE Z,LING G.Dialogue Breakdown Detection using Hierarchical Bi-Directional LSTMs [C]∥Proceedings of the Dialog System Technology Challenges Workshop (DSTC6).Elsevier.Amsterdam.2017.
[21]BENGIO Y,DUCHARME R,VINCENT P,et al.A neural probabilistic language model[J].Journal of machine learning research,2003,3(2):1137-1155.
[22]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013.
[23]ZHANG X,ZHAO J,LECUN Y.Character-level convolutional networks for text classification [C]∥Advances in Neural Infor- mation Processing Systems.New York:Curran Associates,2015:649-657.
[24]ZHANG X,LECUN Y.Which Encoding is the Best for Text Classification in Chinese,English,Japanese and Korean?[J].arXiv:1708.02657,2017.
[25]SORDONI A,BENGIO Y,VAHABI H,et al.A hierarchical recurrent encoder-decoder for generative context-aware query suggestion [C]∥Proceedings of the 24th ACM International on Conference on Information and Knowledge Management.New York:ACM,2015:553-562.
[26]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[27]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[28]CARUNA R.Multitask learning:A knowledge based source of inductive bias[C]∥Machine Learning:Proceedings of the Tenth International Conference.New York:ACM,1993:41-48.
[29]LIU P,QIU X,HUANG X.Recurrent neural network for text classification with multi-task learning[J].arXiv:1605.05101,2016.
[30]SØGAARD A,GOLDBERG Y.Deep multi-task learning with low level tasks supervised at lower layers[C]∥Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.ACL:Stroudsburg.2016:231-235.
[31]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[32]WILLIAMS J D.Web-style ranking and SLU combination for dialog state tracking [C]∥Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL).Stroudsburg:ACL,2014:282-291.
[33]WANG Y,SHEN Y,JIN H.A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling[C]∥Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics Stroudsburg:ACP,2018:309-314.

相关文章 13

[1]	黄少滨, 孙雪薇, 李熔盛. 基于跨句上下文信息的神经网络关系分类方法 Relation Classification Method Based on Cross-sentence Contextual Information for Neural Network 计算机科学, 2022, 49(6A): 119-124. https://doi.org/10.11896/jsjkx.210600150
[2]	郝志峰, 廖祥财, 温雯, 蔡瑞初. 基于多上下文信息的协同过滤推荐算法 Collaborative Filtering Recommendation Algorithm Based on Multi-context Information 计算机科学, 2021, 48(3): 168-173. https://doi.org/10.11896/jsjkx.200700101
[3]	晏旭, 马帅, 曾凤娇, 郭正华, 伍俊龙, 杨平, 许冰. 基于编码-解码器架构的光场深度估计方法 Light Field Depth Estimation Method Based on Encoder-decoder Architecture 计算机科学, 2021, 48(10): 212-219. https://doi.org/10.11896/jsjkx.200900005
[4]	马海江. 基于卷积神经网络与约束概率矩阵分解的推荐算法 Recommendation Algorithm Based on Convolutional Neural Network and Constrained Probability Matrix Factorization 计算机科学, 2020, 47(6A): 540-545. https://doi.org/10.11896/JsJkx.191000172
[5]	杨少鹏, 刘宏哲, 王雪峤. 基于特征图融合的小尺寸人脸检测 Small Size Face Detection Based on Feature Map Fusion 计算机科学, 2020, 47(6): 126-132. https://doi.org/10.11896/jsjkx.19050002
[6]	周鹏程,龚声蓉,钟珊,包宗铭,戴兴华. 基于深度特征融合的图像语义分割 Image Semantic Segmentation Based on Deep Feature Fusion 计算机科学, 2020, 47(2): 126-134. https://doi.org/10.11896/jsjkx.190100119
[7]	赵鹏, 吴礼发, 洪征. 基于经纪人的多云访问控制模型研究 Research on Broker Based Multicloud Access Control Model 计算机科学, 2019, 46(11): 123-129. https://doi.org/10.11896/jsjkx.190300112
[8]	文俊浩,孙光辉,李顺. 基于用户聚类和移动上下文的矩阵分解推荐算法研究 Study on Matrix Factorization Recommendation Algorithm Based on User Clustering and Mobile Context 计算机科学, 2018, 45(4): 215-219. https://doi.org/10.11896/j.issn.1002-137X.2018.04.036
[9]	谌国风,孔俊俊,郭耀,陈向群. 一种智能手机上下文信息获取的代价模型及其应用 Context Retrieval Cost Model on Smartphones and its Application 计算机科学, 2014, 41(11): 132-136. https://doi.org/10.11896/j.issn.1002-137X.2014.11.026
[10]	田宣，李冬梅. 上下文信息检索研究综述 Survey on Contextual Information Retrieval 计算机科学, 2011, 38(9): 18-24.
[11]	. 目标独立的Prolog程序路径依赖分析语义计算机科学, 2008, 35(2): 246-252.
[12]	姚寒冰胡和平卢正鼎李瑞轩. 基于角色和上下文的动态网格访问控制研究计算机科学, 2006, 33(1): 41-44.
[13]	张仰森曹元大. 基于语料库的自然语言建模方法研究计算机科学, 2004, 31(5): 176-179.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed