Computer Science ›› 2024, Vol. 51 ›› Issue (6A): 230800055-7.doi: 10.11896/jsjkx.230800055

• Artificial Intelligenc • Previous Articles     Next Articles

Personalized Dialogue Response Generation Combined with Conversation State Information

GUI Haitao, WANG Zhongqing   

  1. School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215006,China
  • Published:2024-06-06
  • About author:GUI Haitao,born in 1999,postgra-duate,is a member of CCF(No.J1578G).His main research interests include dialogue response generation and so on.
    WANG Zhongqing,born in 1987,Ph.D,associate professor.His main research interests include Natural Language processing,sentiment analysis and informa-tion extraction.
  • Supported by:
    National Natural Science Foundation of China(61806137,61702149).

Abstract: Despite the significant achievements in personalized response generation models,existing studies have not adequately considered the impact of dialogue state information on personalized dialogue responses.To address this issue,this paper proposes a self-supervised dialogue response generation model that incorporates dialogue state to effectively generate personalized replies based on pre-trained generative models.Firstly,we integrate the dialogue state into a situational comedy dataset to enhance the model’s contextual understanding.Secondly,we employ self-supervised training techniques to imbue the pre-trained language ge-neration model with unique dialogue text features and employ various masking strategies to combine dialogue text and dialogue state,further enhancing model performance.Lastly,leveraging historical dialogues,we utilize the self-supervised generative model to produce personalized responses.Experimental results on a self-collected situational comedy dataset demonstrate that the dialogue response generation model incorporating dialogue state outperforms several strong baselines across multiple metrics,thus validating the effectiveness of incorporating dialogue state in personalized response generation models.

Key words: Dialogue response, Conversation state, Self-supervision, Pre-training, Text generation

CLC Number: 

  • TP391
[1]RITTER A,CHERRY C,DOLAN W B.Data-Driven Response Generation in Social Media[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing.Edinburgh,Scotland,UK.:Association for Computational Linguistics,2011:583-593.
[2]LI J,MONROE W,RITTER A,et al.Deep ReinforcementLearning for Dialogue Generation[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Austin,Texas:Association for Computational Linguistics,2016:1192-1202.
[3]PARTHASARATHI P,PINEAU J.Extending Neural Generative Conversational Model using External Knowledge Sources[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels,Belgium:Association for Computational Linguistics,2018:690-695.
[4]YOUNG S J.Probabilistic methods in spoken-dialogue systems[J].Philosophical Transactions of the Royal Society of London.Series A:Mathematical,Physical and Engineering Sciences,2000,358(1769):1389-1402.
[5]YOUNG S,GAŠIĆ M,THOMSON B,et al.POMDP-Based Statistical Spoken Dialog Systems:A Review[J].Proceedings of the IEEE,2013,101(5):1160-1179.
[6]SERBAN I V,LOWE R,HENDERSON P,et al.A Survey ofAvailable Corpora for Building Data-Driven Dialogue Systems[J].arXiv:1512.05742,2017.
[7]LOWE R,POW N,SERBAN I,et al.The Ubuntu Dialogue Corpus:A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems[C]//Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue.Prague,Czech Republic:Association for Computational Linguistics,2015:285-294.
[8]KADLEC R,SCHMID M,KLEINDIENST J.Improved Deep Learning Baselines for Ubuntu Corpus Dialogs[J].arXiv:1510.03753,2015.
[9]GU J C,LING Z H,LIU Q.Interactive matching network for multi-turn response selection in retrieval-based chatbots[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.2019:2321-2324.
[10]WHANG T,LEE D,LEE C,et al.An Effective Domain Adaptive Post-Training Method for BERT in Response Selection[C]//Interspeech 2020.ISCA,2020:1585-1589.
[11]LAN T,CAI D,WANG Y,et al.Exploring Dense Retrieval for Dialogue Response Selection[J].ACM Transactions on Information Systems,2023,42(3):1-29.
[12]NOGUEIRA R,LIN J.From doc2query to docTTTTTquery[J/OL].https://cs.uwaterloo.ca/~jimmylin/publications/Nogueira_Lin_2019_docTTTTTquery-v2.pdf.
[13]YANG W,LU K,YANG P,et al.Critically Examining the“Neural Hype”:Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York,NY,USA:Association for Computing Machinery,2019:1129-1132.
[14]MAIRESSE F,WALKER M.PERSONAGE:Personality Generation for Dialogue[C]//Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics.Prague,Czech Republic:Association for Computational Linguistics,2007:496-503.
[15]LI J,GALLEY M,BROCKETT C,et al.A Diversity-Promoting Objective Function for Neural Conversation Models[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.San Diego,California:Association for Computational Linguistics,2016:110-119.
[16]WOLF T,SANH V,CHAUMOND J,et al.TransferTransfo:A Transfer Learning Approach for Neural Network Based Conversational Agents[J].arXiv:1901.08149,2019.
[17]ZHENG Y,ZHANG R,HUANG M,et al.A Pre-TrainingBased Personalized Dialogue Generation Model with Persona-Sparse Data[J].Proceedings of the AAAI Conference on Artificial Intelligence,2020,34(5):9693-9700.
[18]LIU Q,CHEN Y,CHEN B,et al.You Impress Me:Dialogue Generation via Mutual Persona Perception[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Online:Association for Computational Linguistics,2020:1417-1427.
[19]CHEN N,WANG Y,JIANG H,et al.What would Harry say? Building Dialogue Agents for Characters in a Story[J].arXiv:2211.06869,2022.
[20]WU Y,MA X,YANG D.Personalized Response Generation via Generative Split Memory Network[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Online:Association for Computational Linguistics,2021:1956-1970.
[21]CHEN W,GONG Y,WANG S,et al.DialogVED:A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics(Volume 1 Long Papers).2022:4852-4864.
[22]THOPPILAN R,DE FREITAS D,HALL J,et al.LaMDA:Language Models for Dialog Applications[J].arXiv:2201.08239,2022.
[23]RAFFEL C,SHAZEER N,ROBERTS A,et al.Exploring the limits of transfer learning with a unified text-to-text transformer[J].The Journal of Machine Learning Research,2020,21(1):140:5485-140:5551.
[24]LI Y,SU H,SHEN X,et al.DailyDialog:A Manually LabelledMulti-turn Dialogue Dataset[C]//Proceedings of the Eighth International Joint Conference on Natural Language Processing(Volume 1:Long Papers).Asian Federation of Natural Language Processing,2017:986-995.
[25]SERBAN I,SORDONI A,BENGIO Y,et al.Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models[J].arXiv:1507.04808,2015.
[26]PAPINENI K,ROUKOS S,WARD T,et al.Bleu:a Method for Automatic Evaluation of Machine Translation[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.Philadelphia,Pennsylvania,USA:Association for Computational Linguistics,2002:311-318.
[27]ZHANG W,DENG Y,LI X,et al.Aspect Sentiment Quad Prediction as Paraphrase Generation[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Proces-sing.Online and Punta Cana,Dominican Republic:Association for Computational Linguistics,2021:9209-9219.
[28]LUONG T,PHAM H,MANNING C D.Effective Approaches to Attention-based Neural Machine Translation[C]//Procee-dings of the 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal:Association for Computational Linguistics,2015:1412-1421.
[29]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is All you Need[C]//Proceedings of the 31st International Conference onNeural Information Processing Systems.2017:6000-6010.
[30]ZHANG Y,SUN S,GALLEY M,et al.DIALOGPT:Large-Scale Generative Pre-training for Conversational Response Generation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics:System Demonstrations.Online:Association for Computational Linguistics,2020:270-278.
[31]LI J,GALLEY M,BROCKETT C,et al.A Persona-Based Neural Conversation Model[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Vo-lume 1:Long Papers).Berlin,Germany:Association for Computational Linguistics,2016:994-1003.
[32]LEWIS M,LIU Y,GOYAL N,et al.BART:Denoising Se-quence-to-Sequence Pre-training for Natural Language Generation,Translation,and Comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Online:Association for Computational Linguistics,2020:7871-7880.
[33]SONG H,ZHANG W N,CUI Y,et al.Exploiting Persona Information for Diverse Generation of Conversational Responses[C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence.Macao,China:International Joint Conferences on Artificial Intelligence Organization,2019:5190-5196.
[1] DING Yi, WANG Zhongqing. Study on Pre-training Tasks for Multi-document Summarization [J]. Computer Science, 2024, 51(6A): 230300160-8.
[2] SHI Jiyun, ZHANG Chi, WANG Yuqiao, LUO Zhaojing, ZHANG Meihui. Generation of Structured Medical Reports Based on Knowledge Assistance [J]. Computer Science, 2024, 51(6): 317-324.
[3] CHEN Wenzhong, CHEN Hongmei, ZHOU Lihua, FANG Yuan. Time-aware Pre-training Method for Sequence Recommendation [J]. Computer Science, 2024, 51(5): 45-53.
[4] ZHANG Zhiyuan, ZHANG Weiyan, SONG Yuqiu, RUAN Tong. Multilingual Event Detection Based on Cross-level and Multi-view Features Fusion [J]. Computer Science, 2024, 51(5): 208-215.
[5] WU Jiawei, FANG Quan, HU Jun, QIAN Shengsheng. Pre-training of Heterogeneous Graph Neural Networks for Multi-label Document Classification [J]. Computer Science, 2024, 51(1): 143-149.
[6] TANG Jia, GUO Yan, YE Mingwei, WU Guixing. Multimodal Pre-training Method for Multi-view Contrastive Learning and Semantic Enhancement [J]. Computer Science, 2024, 51(1): 168-174.
[7] YI Liu, GENG Xinyu, BAI Jing. Hierarchical Multi-label Text Classification Algorithm Based on Parallel Convolutional Network Information Fusion [J]. Computer Science, 2023, 50(9): 278-286.
[8] CAI Haoran, YANG Jian, YANG Lin, LIU Cong. Low-resource Thai Speech Synthesis Based on Alternate Training and Pre-training [J]. Computer Science, 2023, 50(6A): 220800127-5.
[9] WANG Taiyan, PAN Zulie, YU Lu, SONG Jingbin. Binary Code Similarity Detection Method Based on Pre-training Assembly Instruction Representation [J]. Computer Science, 2023, 50(4): 288-297.
[10] LIU Zhe, YIN Chengfeng, LI Tianrui. Chinese Spelling Check Based on BERT and Multi-feature Fusion Embedding [J]. Computer Science, 2023, 50(3): 282-290.
[11] SU Qi, WANG Hongling, WANG Zhongqing. Unsupervised Script Summarization Based on Pre-trained Model [J]. Computer Science, 2023, 50(2): 310-316.
[12] GUO Guangxing, YIN Guimei, LIU Chenxu, DUAN Yonghong, QIANG Yan, WANG Yanfei, WANG Tao. Low-dose CT Reconstruction Algorithm Based on Iterative Asymmetric Blind Spot Network [J]. Computer Science, 2023, 50(12): 221-228.
[13] HE Wenhao, WU Chunjiang, ZHOU Shijie, HE Chaoxin. Study on Short Text Clustering with Unsupervised SimCSE [J]. Computer Science, 2023, 50(11): 71-76.
[14] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[15] CHEN Zhang-hui, XIONG Yun. Stylized Image Captioning Model Based on Disentangle-Retrieve-Generate [J]. Computer Science, 2022, 49(6): 180-186.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!