计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (7): 278-286.doi: 10.11896/jsjkx.230500059

• 人工智能 • 上一篇    下一篇

基于关键词异构图的生成式摘要研究

毛兴静1, 魏勇2, 杨昱睿1, 琚生根1   

  1. 1 四川大学计算机学院 成都 610065
    2 中国电子科技集团公司第三十研究所 成都 610041
  • 收稿日期:2023-05-09 修回日期:2023-08-21 出版日期:2024-07-15 发布日期:2024-07-10
  • 通讯作者: 琚生根(jsg@scu.edu.cn)
  • 作者简介:(mxjcdc@163.com)
  • 基金资助:
    国家自然科学基金重点项目(62137001)

KHGAS:Keywords Guided Heterogeneous Graph for Abstractive Summarization

MAO Xingjing1, WEI Yong2, YANG Yurui1, JU Shenggen1   

  1. 1 College of Computer Science,Sichuan University,Chengdu 610065,China
    2 No.30 Research Institute of CETC,Chengdu 610041,China
  • Received:2023-05-09 Revised:2023-08-21 Online:2024-07-15 Published:2024-07-10
  • About author:MAO Xingjing,born in 1998,postgra-duate,is a member of CCF(No.K9114G).Her main research interests include natural language processing and text summarization.
    JU Shenggen,born in 1970,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.14364S).His main research interests include data mining,natural language processing and knowledge graphs.
  • Supported by:
    Key Program of the National Natural Science Foundation of China(62137001).

PDF (PC)

415

摘要： 生成式摘要是自然语言处理中的重要任务,它帮助人们从海量文本中提取简洁而重要的信息。目前主流的生成式摘要模型是基于深度学习的序列到序列模型,这类模型生成的摘要质量更高。但由于缺乏对原文中关键词和句子之间的依赖关系的关注,现有模型生成的摘要仍然存在语义不明、重要信息含量低等问题。针对这个问题,提出了一种基于关键词异构图的生成式摘要模型。该模型通过从原始文本中提取关键词,将其与句子共同作为输入构建异构图,进而学习关键词和句子之间的依赖关系。文档编码器和图编码器分别用于学习文本知识和异构图中的依赖关系。此外,在解码器中采用分层图注意力机制来提高模型在生成摘要时对显著信息的关注。在CNN/Daily Mail和XSum数据集上进行了充分的实验,实验结果表明,所提模型在ROUGE评价指标上有了显著的提升。进一步的人类评估结果显示,所提模型所生成的摘要比基线模型包含更多的关键信息,并具有更高的可读性。

关键词: 生成式摘要, 关键词, 异构图, 图注意力, 序列到序列模型

Abstract: Abstractive summarization is a crucial task in natural language processing that aims to generate concise and informative summaries from a given text.Deep learning-based sequence-to-sequence models have become the mainstream approach for generating abstractive summaries,achieving remarkable performance gains.However,existing models still suffer from issues such as semantic ambiguity and low information content due to the lack of attention to the dependency relationships between key concepts and sentences in the input text.To address this challenge,the keywords guided heterogeneous graph model for abstractive summarization is proposed.This model leverages extracted keywords and constructs a heterogeneous graph with both keywords and sentences as input to model the dependency relationships between them.A document encoder and a graph encoder are respectively used to capture textual information and dependency relationships in the heterogeneous graph.Moreover,a hierarchical graph attention mechanism is introduced in the decoder to improve the model's attention to significant information when generating summaries..Extensive experiments on the CNN/Daily Mail and XSum datasets demonstrate that the proposed model outperforms existing methods in terms of the ROUGE evaluation metric.Human evaluations also reveal that the generated summaries by the proposed model contain more key information and are more readable compared to the baseline models.

Key words: Abstractive summarization, Keywords, Heterogeneous graph, Graph attention, Sequence to sequence model

中图分类号: 

  • TP391
[1]HEWETT F,STEDE M.Extractive Summarisation for German-language Data:A Text-level Approach with Discourse Features [C]//Proceedings of the 29th International Conference on Computational Linguistics.International Committee on Computational Linguistics,2022:756-765.
[2]LI J,ZHANG C,CHEN X,et al.Survey on Automatic Text Summarization [J].Journal of Computer Research and Development,2021,58(1):1-21.
[3]LIU Y,LIU P,RADEV D,et al.2022.BRIO:Bringing Order to Abstractive Summarization[C]//Proceedings of the 60th An-nual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).Association for Computational Linguistics,2022:2890-2903.
[4]LIU X,WANG H,WU J.GAN and Chinese WordNet Based Text Summarization Technology [J].Computer Science,2022,49(12):301-304.
[5]MIKOLOV T,KARAFIÁT M,BURGET L,et al.Recurrentneural network based language model[C]//INTERSPEECH-11th Annual Conference of the International Speech Communication Association.2010:1045-1048.
[6]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[7]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial networks[J].Communications of the ACM,2020,63(11):139-144.
[8]HE J,KRYSCINSKI W,MCCANN B,et al.CTRLsum:To-wards Generic Controllable Text Summarization[C]//Procee-dings of the 2022 Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics,2022:5879-5915.
[9]LI H,ZHU J,ZHANG J,et al.Keywords-Guided Abstractive Sentence Summarization[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.2020:8196-8203.
[10]CAO Z,WEI F,LI W,et al.Faithful to the Original:Fact Aware Neural Abstractive Summarization[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[11]LI W,XIAO X,LIU J,et al.Leveraging Graph to Improve Abstractive Multi-Document Summarization [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics,2020:6232-6243.
[12]FENG X,FENG X,QIN B.Incorporating CommonsenseKnowledge into Abstractive Dialogue Summarization via Hete-rogeneous Graph Networks[C]//China National Conference on Chinese Computational Linguistics.Springer,CCL 2021:Chinese Computational Linguistics,2021:127-142.
[13]NALLAPATI R,ZHOU B,SANTOS C,et al.Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning.Association for Computa-tional Linguistics,2016:280-290.
[14]NARAYAN S,COHEN S,LAPATA M.Don't Give Me the Details,Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics,2018:1997-1807.
[15]LIN C.ROUGE:A Package for Automatic Evaluation of Summaries[C]//Text Summarization Branches Out.Association for Computational Linguistics,2004:74-81.
[16]SEE A,LIU P,MANNING C.Get To The Point:Summariza-tion with Pointer-Generator Networks[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).Association for Computa-tional Linguistics,2017:1073-1083.
[17]DOU Z,LIU P,HAYASHI H,et al.GSum:A General Framework for Guided Neural Abstractive Summarization[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Association for Computational Linguistics,2021:4830-4842.
[18]DEVLIN J,CHANG M,LEE K,et al.BERT:Pre-training ofDeep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers).Association for Computational Linguistics,2019:4171-4186.
[19]LEWIS M,LIU Y,GOYAL N,et al.BART:Denoising Se-quence-to-Sequence Pre-training for Natural Language Generation,Translation,and Comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics,2020:7871-7880.
[20]LIU Y,OTT M,GOYAL N,et al.Roberta:A robustly opti-mizedbert pretraining approach[EB/OL].(2019-07-26) [2022-07-28].https://doi.org/10.48550/arXiv.1907.11692.
[21]XU W,LI C,LEE M,et al.Multi-task learning for abstractive text summarization with key information guide network[J/OL].EURASIP Journal on Advances in Signal Processing 2020.https://doi.org/10.1186/s13634-020-00674-7.
[22]GUAN S,ZHU P,WEI Z.Knowledge and Keywords Augmented Abstractive Sentence Summarization[C]//Proceedings of the Third Workshop on New Frontiers in Summarization,Online and in Dominican Republic.Association for Computational Linguistics.2021:25-32.
[23]HE J,KRYSCINSKI W,MCCANN B,et al.CTRLsum:To-wards Generic Controllable Text Summarization[C]//Procee-dings of the 2022 Conference on Empirical Methods in Natural Language Processing,Abu Dhabi,United Arab Emirates.Association for Computational Linguistics.2022:5879-5915.
[24]MIHALCEA R,TARAU P.Textrank:Bringing Order into Text [C]//Empirical Methods in Natural Language Processing(EMNLP).ACL,2004:404-411.
[25]ERKAN G,RADEV D R.Lexrank:Graph-based Lexical Centrality as Salience in Text Summarization [J].Journal of Artificial Intelligence Research,2004,22,457-479.
[26]WAN X,XIAO J.Single Document Keyphrase Extraction Using Neighborhood Knowledge [C]//Proceedings of the AAAI Conference on Artificial Intelligence.AAAI,2008:855-860.
[27]KIPF T,WELLING M.Semi-Supervised Classification withGraph Convolutional Networks[C]//Proceedings of the 5th International Conference on Lear-ning Representations.ICLR 2017.
[28]YASUNAGA M,ZHANG R,MEELU K,et al.Graph-basedNeural Multi-Document Summarization[C]//Proceedings of the 21st Conference on Computational Natural Language Learning(CoNLL 2017).Vancouver,Canada:Association for Computational Linguistics,2017:452-462.
[29]WANG D,LIU P,ZHENG Y,et al.Heterogeneous graph neuralnetworks for extractive document summarization[C]//Procee-dings of the 58th Annual Meeting of the Association for Computational Linguistics,Online.Association for Computational Linguistics:2020:6209-6219.
[30]JIN H,WANG T,WAN X.Semsum:Semantic dependencyguided neural abstractive summarization[C]//Proceedings of the AAAI Conference on Artificial Intelligence.AAAI,2020:8026-8033.
[31]GROOTENDORST M.KeyBERT:Minimal keyword extraction with BERT[EB/OL].https://doi.org/10.5281/zenodo.4461265.
[32]LI C,XU W,LI S,et al.Guiding Generation for AbstractiveText Summarization Based on Key Information Guide Network[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 2(Short Papers),New Orleans,Louisiana.Association for Computational Linguistics.2018:55-60.
[33]KEDZIORSKI R,BEKAL D,LUAN Y,et al.Text Generation from Knowledge Graphs with Graph Transformers[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Association for Computational Linguistics,2019:2284-2293.
[34]LIU Y,LAPATA M.Text Summarization with Pretrained Encoders[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).ACL,2019:3730-3740.
[35]SONG K,TAN X,QIN T,et al.MASS:Masked Sequence to Sequence Pre-training for Language Generation[C]//International Conference on Machine Learning.PMLR,2019:5926-5936.
[36]ZHANG J,ZHAO Y,SALEH M,et al.Pegasus:Pre-training with extracted gap sentences for abstractive summarization[C]//Proceedings of the 37th International Conference on Machine Learning(ICML'20).JMLR.org,2020:11328-11339.
[37]JI X,ZHAO W.SKGSUM:Abstractive Document Summarization with Semantic Knowledge Graphs[C]//2021 International Joint Conference on Neural Networks(IJCNN).Shenzhen,China,2021:1-8.
[38]JIANG M,ZOU Y,XU J,et al.GATSum:Graph-Based Topic-Aware Abstract Text Summarization[J].Information Technology and Control,2022:51(2),345-355.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发