Computer Science ›› 2021, Vol. 48 ›› Issue (8): 234-239.doi: 10.11896/jsjkx.200700162

Special Issue: Natural Language Processing

• Artificial Intelligence • Previous Articles     Next Articles

Compound Conversation Model Combining Retrieval and Generation

YANG Hui-min, MA Ting-huai   

  1. College of Computer and Software,Nanjing University of Information Science & Technology,Nanjing 210044,China
  • Received:2020-07-26 Revised:2020-09-17 Published:2021-08-10
  • About author:YANG Hui-min,born in 1997,postgra-duate.Her main research interests include data mining and data sharing.(2432640905@qq.com)MA Ting-huai,born in 1974,Ph.D,professor,is a member of China Computer Federation.His main research interests include data mining,data sharing and privacy protection.
  • Supported by:
    National Natural Science Foundation of China(U1736105).

Abstract: Conversation model is one of the important directions of natural language processing.Today's dialogue models are mainly divided into retrieval-based methods and generation-based methods.However,the retrieval method cannot respond to questions that do not appear in the corpus,and the generation method is prone to problems with safe responses.In view of this,a compound conversation model that combines retrieval and generation is proposed,and the retrieval method and generation method are combined to make up for their shortcomings.First,K retrieval contexts and corresponding K retrieval candidate responses are obtained through the retrieval module.In the multi-response generation module,retrieval contexts are further combined to obtain several generation candidate responses.The candidate response ranking module is divided into two steps:pre-screening and post-reranking.The pre-screening part obtains the optimal retrieval response and the optimal generated response by calculating the similarity between the input question and candidate responses,and the post-reranking part further selects the most suitable answer to the input question.Experimental results show that the BLUE index increased by 6%,and the diversity index increased by 12%.

Key words: Conversation system, Generation model, Post-reranking, Retrieval model, Transformer

CLC Number: 

  • TP319.1
[1]WANG Y,HE Q T.Research on Intelligent Question Answe-ring System[J].Electronic Technology and Software Enginee-ring,2019(5):174-175.
[2]VINYALS O,LE Q.A neural conversational model[J].arXiv:1506.05869,2015.
[3]SHEN Y,HE X,GAO J,et al.A latent semantic model with convolutional-pooling structure for information retrieval[C]//Proceedings of the 23rd ACM International Conference on Information and Knowledge Management.Shanghai,China:ACM,2014:101-110.
[4]WAN S,LAN Y,XU J,et al.Match-srnn:Modeling the recursive matching structure with spatial rnn[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence.New York,USA:Margan Kaufmann,2016:2922-2928.
[5]SUTSKEVER I,VINYALS O,LE Q V.Sequence to sequence learning with neural networks[C]//Advances in Neural Information Processing Systems.Montreal,Quebec,Canada:MIT PRESS,2014:3104-3112.
[6]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[C]//3rd International Conference on Learning Representations.San Diego,USA:ICLR,2015:1-9.
[7]ZHAO Y Y,WANG Z Y,WANG P,et al.A review of task-based dialogue systems[J].Chinese Journal of Computers,2020,43(10),1862-1896.
[8]HORI T,WANG W,KOJI Y,et al.Adversarial training and decoding strategies for end-to-end neural conversation models[J].Computer Speech & Language,2019,54:122-139.
[9]BROMLEY J,GUYON I,LECUN Y,et al.Signature verification using a “siamese” time delay neural network[C]//Advances in Neural Information Processing Systems.1994:737-744.
[10]CHI Z,ZHANG B.A sentence similarity estimation methodbased on improved siamese network[J].Journal of Intelligent Learning Systems and Applications,2018,10(4):121-134.
[11]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems.Long Beach,USA:MIT PRESS,2017:5998-6008.
[12]ZHU Z,LIANG J,LI D,et al.Hot topic detection based on arefined TF-IDF algorithm[J].IEEE Access,2019,7:26996-27007.
[13]GU Y J,GUI X L,LI D F,et al.A Survey of Machine Reading Comprehension Based on Neural Networks[J].Journal of Software,2020,31(7):2095-2126.
[14]PANDEY G,CONTRACTOR D,KUMAR V,et al.Exemplar encoder-decoder for neural conversation generation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Melbourne,Australia:Association for Computational Linguistics,2018:1329-1338.
[15]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[16]PRECHELT L.Automatic early stopping using cross valida-tion:quantifying the criteria[J].Neural Networks,1998,11(4):761-767.
[17]WU Y,WEI F,HUANG S,et al.Response generation by con-text-aware prototype editing[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Honolulu,USA:AAAI,2019:7281-7288.
[18]PAPINENI K,ROUKOS S,WARD T,et al.BLEU:a methodfor automatic evaluation of machine translation[C]//Procee-dings of the 40th Annual Meeting on Association for Computational Linguistics.Philadelphia,USA:Association for Computational Linguistics,2002:311-318.
[19]LI J,GALLEY M,BROCKETT C,et al.A diversity-promoting objective function for neural conversation models[C]//Procee-dings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics.San Diego,USA:Association for Computational Linguistics,2016:110-119.
[20]ZHOU Q A,LI Z J.Improved model and tuning method for na-tural language understanding of task-oriented dialogue system based on BERT[J].Journal of Chinese Information Processing,2020,34(5):82-90.
[1] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[2] KANG Yan, XU Yu-long, KOU Yong-qi, XIE Si-yu, YANG Xue-kun, LI Hao. Drug-Drug Interaction Prediction Based on Transformer and LSTM [J]. Computer Science, 2022, 49(6A): 17-21.
[3] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[4] ZHAO Xiao-hu, YE Sheng, LI Xiao. Multi-algorithm Fusion Behavior Classification Method for Body Bone Information Reconstruction [J]. Computer Science, 2022, 49(6): 269-275.
[5] LU Liang, KONG Fang. Dialogue-based Entity Relation Extraction with Knowledge [J]. Computer Science, 2022, 49(5): 200-205.
[6] HUO Shuai, PANG Chun-jiang. Research on Sentiment Analysis Based on Transformer and Multi-channel Convolutional Neural Network [J]. Computer Science, 2021, 48(6A): 349-356.
[7] YANG Jin-cai, CAO Yuan, HU Quan, SHEN Xian-jun. Relation Classification of Chinese Causal Compound Sentences Based on Transformer Model and Relational Word Feature [J]. Computer Science, 2021, 48(6A): 295-298.
[8] JIANG Qi, SU Wei, XIE Ying, ZHOUHONG An-ping, ZHANG Jiu-wen, CAI Chuan. End-to-End Chinese-Braille Automatic Conversion Based on Transformer [J]. Computer Science, 2021, 48(11A): 136-141.
[9] ZHENG Wen-ping, QU Rui and MU Jun-fang. Generation Algorithm for Scale-free Networks with Community Structure [J]. Computer Science, 2018, 45(2): 76-83.
[10] LI Feng and XIA Li. Transformer Fault Monitoring Expert System Based on Rule Base [J]. Computer Science, 2016, 43(Z11): 564-567.
[11] CHEN Jing, LIU Yan and WANG Xu-zhong. Group Partition in Topic-related Microblogging Spreading Based on Probability Generation Model [J]. Computer Science, 2016, 43(8): 223-228.
[12] . Contextual Preferences-based Approach for X1VIL Query Results Ranking [J]. Computer Science, 2012, 39(Z6): 271-276.
[13] TIAN Xuan LI Kong-mei. Survey on Contextual Information Retrieval [J]. Computer Science, 2011, 38(9): 18-24.
[14] LUAN Xi-dao,XIE Yu-xiang,TAN Yi-hong,CHEN Zhi-ping,ZHAO Bi-hai,HU Sai. Research Development of Multimedia Semantic Model [J]. Computer Science, 2010, 37(11): 1-6.
[15] . [J]. Computer Science, 2008, 35(7): 157-160.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!