计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (11): 192-200.doi: 10.11896/jsjkx.230300241

• 人工智能 • 上一篇    下一篇

基于多粒度对比学习的聊天对话摘要模型

康梦瑶1,2, 刘扬1,2, 黄俊恒1,2, 王佰玲1,2, 刘树龙1   

  1. 1 哈尔滨工业大学(威海)计算机科学与技术学院 山东 威海 264209
    2 哈尔滨工业大学(威海)网络空间安全研究院 山东 威海264209
  • 收稿日期:2023-03-31 修回日期:2023-05-09 出版日期:2023-11-15 发布日期:2023-11-06
  • 通讯作者: 王佰玲(wbl@hit.edu.cn)
  • 作者简介:(kangmengyao99@163.com)
  • 基金资助:
    国家重点研发计划(2020YFB2009502);国家自然科学基金(62272129);中央高校基本科研业务费专项资金(HIT.NSRIF.2020098)

Chat Dialogue Summary Model Based on Multi-granularity Contrastive Learning

KANG Mengyao1,2, LIU Yang1,2, HUANG Junheng1,2, WANG Bailing1,2, LIU Shulong1   

  1. 1 School of Computer Science and Technology,Harbin Institute of Technology(Weihai),Weihai,Shandong 264209,China
    2 Research Institute of Cyberspace Security,Harbin Institute of Technology(Weihai),Weihai,Shandong 264209,China
  • Received:2023-03-31 Revised:2023-05-09 Online:2023-11-15 Published:2023-11-06
  • About author:KANG Mengyao,born in 1999,postgraduate.Her main research interests include artificial intelligence,natural language processing and cyber security.WANG Bailing,born in 1978,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include cyber security and industrial Internet security.
  • Supported by:
    National Key R & D Program of China(2020YFB2009502),National Natural Science Foundation of China(62272129) and Fundamental Research Funds for the Central Universities(HIT.NSRIF.2020098).

PDF (PC)

2538

摘要： 社交网络的发展在给人们带来便捷的同时也产生了海量的聊天数据,如何从聊天对话中筛选出关键信息成为一大难题。聊天摘要是解决此类问题的有效工具,既不必重复浏览冗长的聊天记录,又可以快速获取重要内容。目前,预训练模型被广泛应用于各种类型的文本,包括非结构化、半结构化和结构化文本。然而,针对聊天对话文本的应用,常见的预训练模型难以捕捉到其独特的结构特征,仍需进一步探索与改进。对此,提出了一种基于对比学习的聊天摘要算法MGCSum。该算法无需人工标注数据集,便于学习和迁移。首先使用文档频数、词项频数和信息熵构造了针对聊天文本的停用词列表,去除聊天中的干扰信息;其次,从词语和主题两个粒度进行自监督对比学习,识别对话中的结构信息,挖掘聊天中的关键词和不同主题信息。在聊天摘要公开数据集SAMSum和金融欺诈对话数据集FINSum上进行实验,结果表明,与当前主流的聊天摘要方法相比,该算法在摘要的连贯性、信息量和ROUGE评价指标上均有显著提升。

关键词: 聊天摘要, 对比学习, 预训练模型, 关键词检测, 主题分割

Abstract: While the development of social networks brings convenience,but also generates massive amounts of chat data.How to filter key information from chat conversations has become a major difficulty.Chat summary is an effective tool to solve such pro-blems,as it allows users to quickly obtain important content without having to repeatedly browse through lengthy chat records.Currently,pre-trained models are widely used in various types of text,including unstructured,semi-structured,and structured text.However,for chat dialogue text,common pre-trained models are often unable to capture its unique structural features,and further exploration and improvement are still needed.To address these issues,this paper proposes a chat summary model MGCSum,which based on multi-granularity contrastive learning and does not require manual annotation of the datasets,making it easy to learn and transfer.Firstly,a stop word list for chat text is constructed by using document frequency,term frequency and entropy to remove interference information in chat.Then,self-supervised contrastive learning is performed at the granularity of words and topics to identify the structure of conversation,uncover keywords and distinct topic information in chats.Experimental results on the publicly available chat summary datasets SAMSum and financial fraud dialogue summary dataset FINSum show that,compared to current mainstream chat summary methods,this algorithm significantly improves coherence,information content and ROUGE evaluation metrics.

Key words: Chat summary, Contrastive learning, Pre-trained models, Keyword detection, Topic segmentation

中图分类号: 

  • TP391
[1]中国互联网络信息中心.第51次《中国互联网络发展状况统计报告》.[EB/OL].https://www.cnnic.cn/n4/2023/0302/c199-10755.html.
[2]KRYSCINSKI W,KESKAR N S,MCCANN B,et al.NeuralText Summarization:A Critical Evaluation[C]//Conference on Empirical Methods in Natural Language Processing & International Joint Conference on Natural Language Processing.2019:540-551.
[3]KOTO F.A publicly available Indonesian corpora for automatic abstractive and extractive chat summarization[C]//Interna-tional Conference on Language Resources and Evaluation.2016:801-805.
[4]ZOU Y,LIN J,ZHAO L,et al.Unsupervised summarization for chat logs with topic-oriented ranking and context-aware auto-encoders[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021,35(16):14674-14682.
[5]SHANG G,DING W,ZHANG Z,et al.Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.2018:664-674.
[6]ZHENG J,ZHAO Z,SONG Z,et al.Abstractive meeting summarization by hierarchical adaptive segmental network learning with multiple revising steps [J].Neurocomputing,2020,378:179-188.
[7]ZHU C,XU R,ZENG M,et al.End-to-end abstractive summarization for meetings [J].arXiv:2004.02016,2020.
[8]GLIWA B,MOCHOL I,BIESEK M,et al.SAMSum Corpus:AHuman-annotated Dialogue Dataset for Abstractive Summarization[C]//Workshop on New Frontiers in Summarization.2019:70-79.
[9]CHEN J,YANG D.Multi-View Sequence-to-Sequence Modelswith Conversational Structure for Abstractive Dialogue Summarization[C]//Conference on Empirical Methods in Natural Language Processing.2020:4106-4118.
[10]CHEN J,YANG D.Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs[C]//Confe-rence of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2021:1380-1391.
[11]WU C S,LIU L,LIU W,et al.Controllable abstractive dialogue summarization with sketch supervision[J].arXiv:2105.14064,2021.
[12]ZHAO L,XU W,GUO J.Improving abstractive dialogue summarization with graph structures and topic words[C]//International Conference on Computational Linguistics.2020:437-449.
[13]FANG H,WANG S,ZHOU M,et al.Cert:Contrastive self-supervised learning for language understanding[J].arXiv:2005.12766,2020.
[14]KLEIN T,NABI M.Contrastive Self-Supervised Learning forCommonsense Reasoning[C]//Annual Meeting of the Association for Computational Linguistics.2020:7517-7523.
[15]GUNEL B,DU J,CONNEAU A,et al.Supervised contrastivelearning for pre-trained language model fine-tuning[J].arXiv:2011.01403,2020.
[16]GAO T,YAO X,CHEN D.SimCSE:Simple Contrastive Lear-ning of Sentence Embeddings[C]//Conference on Empirical Methods in Natural Language Processing,Association for Computational Linguistics.2021:6894-6910.
[17]SCHROFF F,KALENICHENKO D,PHILBIN J.Facenet:Aunified embedding for face recognition and clustering[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:815-823.
[18]XU S,ZHANG X,WU Y,et al.Sequence level contrastivelearning for text summarization[C]//Conference on Artificial Intelligence.2022:11556-11565.
[19]ZHONG M,LIU P,CHEN Y,et al.Extractive Summarization as Text Matching[C]//Annual Meeting of the Association for Computational Linguistics.2020:6197-6208.
[20]WU H,MA T,WU L,et al.Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning[C]//Confe-rence on Empirical Methods in Natural Language Processing.2020:3612-3621.
[21]LIU Y,LIU P.SimCLS:A Simple Framework for ContrastiveLearning of Abstractive Summarization[C]//Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing.2021:1065-1072.
[22]LIU J,ZOU Y,ZHANG H,et al.Topic-Aware ContrastiveLearning for Abstractive Dialogue Summarization[C]//Asso-ciation for Computational Linguistics:EMNLP.2021:1229-1243.
[23]GENG Z,ZHONG M,YIN Z,et al.Improving Abstractive Dialogue Summarization with Speaker-Aware Supervised Contrastive Learning[C]//International Conference on Computational Linguistics.2022:6540-6546.
[24]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[25]LEWIS M,LIU Y,GOYAL N,et al.Bart:Denoising sequence-to-sequence pre-training for natural language generation,translation,and comprehension[J].arXiv:1910.13461,2019.
[26]SINKA M P,CORNE D W.Towards modernised and web-specific stoplists for web document analysis[C]//International Conference on Web Intelligence.IEEE,2003:396-402.
[27]LIN C Y.Rouge:A package for automatic evaluation of summaries[C]//Proceedings of the Workshop on Text Summarization Branches Out.2004:74-81.
[28]ZHANG Y,SUN S,GALLEY M,et al.Dialogpt:Large-scalegenerative pre-training for conversational response generation[J].arXiv:1911.00536,2019.
[29]WU F,FAN A,BAEVSKI A,et al.Pay less attention withlightweight and dynamic convolutions[J].arXiv:1901.10430,2019.
[30]CHEN Y C,BANSAL M.Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting[C]//Annual Meeting of the Association for Computational Linguistics.2018:675-686.
[31]FENG X,FENG X,QIN B.Incorporating commonsense know-ledge into abstractive dialogue summarization via heterogeneous graph networks[C]//Chinese Computational Linguistics:20th China National Conference.Cham:Springer International Publishing,2021:127-142.
[32]DONG L,YANG N,WANG W,et al.Unified language model pre-training for natural language understanding and generation[J].Advances in Neural Information Processing Systems,2019,32:13063-13075.
[33]KIM S,JOO S J,CHAE H,et al.Mind the Gap! Injecting Commonsense Knowledge for Abstractive Dialogue Summarization[J].arXiv:2209.00930,2022.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发