Computer Science ›› 2023, Vol. 50 ›› Issue (11): 192-200.doi: 10.11896/jsjkx.230300241

• Artificial Intelligence • Previous Articles     Next Articles

Chat Dialogue Summary Model Based on Multi-granularity Contrastive Learning

KANG Mengyao1,2, LIU Yang1,2, HUANG Junheng1,2, WANG Bailing1,2, LIU Shulong1   

  1. 1 School of Computer Science and Technology,Harbin Institute of Technology(Weihai),Weihai,Shandong 264209,China
    2 Research Institute of Cyberspace Security,Harbin Institute of Technology(Weihai),Weihai,Shandong 264209,China
  • Received:2023-03-31 Revised:2023-05-09 Online:2023-11-15 Published:2023-11-06
  • About author:KANG Mengyao,born in 1999,postgraduate.Her main research interests include artificial intelligence,natural language processing and cyber security.WANG Bailing,born in 1978,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include cyber security and industrial Internet security.
  • Supported by:
    National Key R & D Program of China(2020YFB2009502),National Natural Science Foundation of China(62272129) and Fundamental Research Funds for the Central Universities(HIT.NSRIF.2020098).

Abstract: While the development of social networks brings convenience,but also generates massive amounts of chat data.How to filter key information from chat conversations has become a major difficulty.Chat summary is an effective tool to solve such pro-blems,as it allows users to quickly obtain important content without having to repeatedly browse through lengthy chat records.Currently,pre-trained models are widely used in various types of text,including unstructured,semi-structured,and structured text.However,for chat dialogue text,common pre-trained models are often unable to capture its unique structural features,and further exploration and improvement are still needed.To address these issues,this paper proposes a chat summary model MGCSum,which based on multi-granularity contrastive learning and does not require manual annotation of the datasets,making it easy to learn and transfer.Firstly,a stop word list for chat text is constructed by using document frequency,term frequency and entropy to remove interference information in chat.Then,self-supervised contrastive learning is performed at the granularity of words and topics to identify the structure of conversation,uncover keywords and distinct topic information in chats.Experimental results on the publicly available chat summary datasets SAMSum and financial fraud dialogue summary dataset FINSum show that,compared to current mainstream chat summary methods,this algorithm significantly improves coherence,information content and ROUGE evaluation metrics.

Key words: Chat summary, Contrastive learning, Pre-trained models, Keyword detection, Topic segmentation

CLC Number: 

  • TP391
[1]中国互联网络信息中心.第51次《中国互联网络发展状况统计报告》.[EB/OL].https://www.cnnic.cn/n4/2023/0302/c199-10755.html.
[2]KRYSCINSKI W,KESKAR N S,MCCANN B,et al.NeuralText Summarization:A Critical Evaluation[C]//Conference on Empirical Methods in Natural Language Processing & International Joint Conference on Natural Language Processing.2019:540-551.
[3]KOTO F.A publicly available Indonesian corpora for automatic abstractive and extractive chat summarization[C]//Interna-tional Conference on Language Resources and Evaluation.2016:801-805.
[4]ZOU Y,LIN J,ZHAO L,et al.Unsupervised summarization for chat logs with topic-oriented ranking and context-aware auto-encoders[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021,35(16):14674-14682.
[5]SHANG G,DING W,ZHANG Z,et al.Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.2018:664-674.
[6]ZHENG J,ZHAO Z,SONG Z,et al.Abstractive meeting summarization by hierarchical adaptive segmental network learning with multiple revising steps [J].Neurocomputing,2020,378:179-188.
[7]ZHU C,XU R,ZENG M,et al.End-to-end abstractive summarization for meetings [J].arXiv:2004.02016,2020.
[8]GLIWA B,MOCHOL I,BIESEK M,et al.SAMSum Corpus:AHuman-annotated Dialogue Dataset for Abstractive Summarization[C]//Workshop on New Frontiers in Summarization.2019:70-79.
[9]CHEN J,YANG D.Multi-View Sequence-to-Sequence Modelswith Conversational Structure for Abstractive Dialogue Summarization[C]//Conference on Empirical Methods in Natural Language Processing.2020:4106-4118.
[10]CHEN J,YANG D.Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs[C]//Confe-rence of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2021:1380-1391.
[11]WU C S,LIU L,LIU W,et al.Controllable abstractive dialogue summarization with sketch supervision[J].arXiv:2105.14064,2021.
[12]ZHAO L,XU W,GUO J.Improving abstractive dialogue summarization with graph structures and topic words[C]//International Conference on Computational Linguistics.2020:437-449.
[13]FANG H,WANG S,ZHOU M,et al.Cert:Contrastive self-supervised learning for language understanding[J].arXiv:2005.12766,2020.
[14]KLEIN T,NABI M.Contrastive Self-Supervised Learning forCommonsense Reasoning[C]//Annual Meeting of the Association for Computational Linguistics.2020:7517-7523.
[15]GUNEL B,DU J,CONNEAU A,et al.Supervised contrastivelearning for pre-trained language model fine-tuning[J].arXiv:2011.01403,2020.
[16]GAO T,YAO X,CHEN D.SimCSE:Simple Contrastive Lear-ning of Sentence Embeddings[C]//Conference on Empirical Methods in Natural Language Processing,Association for Computational Linguistics.2021:6894-6910.
[17]SCHROFF F,KALENICHENKO D,PHILBIN J.Facenet:Aunified embedding for face recognition and clustering[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:815-823.
[18]XU S,ZHANG X,WU Y,et al.Sequence level contrastivelearning for text summarization[C]//Conference on Artificial Intelligence.2022:11556-11565.
[19]ZHONG M,LIU P,CHEN Y,et al.Extractive Summarization as Text Matching[C]//Annual Meeting of the Association for Computational Linguistics.2020:6197-6208.
[20]WU H,MA T,WU L,et al.Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning[C]//Confe-rence on Empirical Methods in Natural Language Processing.2020:3612-3621.
[21]LIU Y,LIU P.SimCLS:A Simple Framework for ContrastiveLearning of Abstractive Summarization[C]//Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing.2021:1065-1072.
[22]LIU J,ZOU Y,ZHANG H,et al.Topic-Aware ContrastiveLearning for Abstractive Dialogue Summarization[C]//Asso-ciation for Computational Linguistics:EMNLP.2021:1229-1243.
[23]GENG Z,ZHONG M,YIN Z,et al.Improving Abstractive Dialogue Summarization with Speaker-Aware Supervised Contrastive Learning[C]//International Conference on Computational Linguistics.2022:6540-6546.
[24]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[25]LEWIS M,LIU Y,GOYAL N,et al.Bart:Denoising sequence-to-sequence pre-training for natural language generation,translation,and comprehension[J].arXiv:1910.13461,2019.
[26]SINKA M P,CORNE D W.Towards modernised and web-specific stoplists for web document analysis[C]//International Conference on Web Intelligence.IEEE,2003:396-402.
[27]LIN C Y.Rouge:A package for automatic evaluation of summaries[C]//Proceedings of the Workshop on Text Summarization Branches Out.2004:74-81.
[28]ZHANG Y,SUN S,GALLEY M,et al.Dialogpt:Large-scalegenerative pre-training for conversational response generation[J].arXiv:1911.00536,2019.
[29]WU F,FAN A,BAEVSKI A,et al.Pay less attention withlightweight and dynamic convolutions[J].arXiv:1901.10430,2019.
[30]CHEN Y C,BANSAL M.Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting[C]//Annual Meeting of the Association for Computational Linguistics.2018:675-686.
[31]FENG X,FENG X,QIN B.Incorporating commonsense know-ledge into abstractive dialogue summarization via heterogeneous graph networks[C]//Chinese Computational Linguistics:20th China National Conference.Cham:Springer International Publishing,2021:127-142.
[32]DONG L,YANG N,WANG W,et al.Unified language model pre-training for natural language understanding and generation[J].Advances in Neural Information Processing Systems,2019,32:13063-13075.
[33]KIM S,JOO S J,CHAE H,et al.Mind the Gap! Injecting Commonsense Knowledge for Abstractive Dialogue Summarization[J].arXiv:2209.00930,2022.
[1] XU Jie, WANG Lisong. Contrastive Clustering with Consistent Structural Relations [J]. Computer Science, 2023, 50(9): 123-129.
[2] HU Shen, QIAN Yuhua, WANG Jieting, LI Feijiang, LYU Wei. Super Multi-class Deep Image Clustering Model Based on Contrastive Learning [J]. Computer Science, 2023, 50(9): 192-201.
[3] LI Xiang, FAN Zhiguang, LIN Nan, CAO Yangjie, LI Xuexiang. Self-supervised Learning for 3D Real-scenes Question Answering [J]. Computer Science, 2023, 50(9): 220-226.
[4] WANG Mingxia, XIONG Yun. Disease Diagnosis Prediction Algorithm Based on Contrastive Learning [J]. Computer Science, 2023, 50(7): 46-52.
[5] WU Jufeng, ZHAO Xungang, ZHOU Qiang, RAO Ning. Contrastive Learning for Low-light Image Enhancement [J]. Computer Science, 2023, 50(6A): 220600171-6.
[6] HE Chao, CHEN Jinjie, JIN Zhao, LEI Yinjie. Automatic Modulation Recognition Method Based on Multimodal Time-Frequency Feature Fusion [J]. Computer Science, 2023, 50(4): 226-232.
[7] WU Yushan, XU Zengmin, ZHANG Xuelian, WANG Tao. Self-supervised Action Recognition Based on Skeleton Data Augmentation and Double Nearest Neighbor Retrieval [J]. Computer Science, 2023, 50(11): 97-106.
[8] LIU Nan, ZHANG Fengli, YIN Jiaqi, CHEN Xueqin, WANG Ruijin. Rumor Detection Model on Social Media Based on Contrastive Learning with Edge-inferenceAugmentation [J]. Computer Science, 2023, 50(11): 49-54.
[9] XIE Zhuo, KANG Le, ZHOU Lijuan, ZHANG Zhihong. Clustering Method Based on Contrastive Learning for Multi-relation Attribute Graph [J]. Computer Science, 2023, 50(11): 62-70.
[10] ZHENG Shijie, WANG Gaocai. Study on Fine-grained Image Classification Based on ConvNeXt Heatmap Localization and Contrastive Learning [J]. Computer Science, 2023, 50(10): 119-125.
[11] XIAO Yang, QIN Jianyang, LI Kenli, WANG Ge, LI Rui, LIAO Qing. Co-Forecasting for Multi-modal Traffic Flow Based on Graph Contrastive Learning [J]. Computer Science, 2023, 50(10): 135-145.
[12] YUAN De-sen, LIU Xiu-jing, WU Qing-bo, LI Hong-liang, MENG Fan-man, NGAN King-ngi, XU Lin-feng. Visual Question Answering Method Based on Counterfactual Thinking [J]. Computer Science, 2022, 49(12): 229-235.
[13] HUANG Xin, LEI Gang, CAO Yuan-long, LU Ming-ming. Review on Interactive Question Answering Techniques Based on Deep Learning [J]. Computer Science, 2021, 48(12): 286-296.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!