计算机科学 ›› 2026, Vol. 53 ›› Issue (6): 50-58.doi: 10.11896/jsjkx.250600151
上怡, 英迪, 赵晖
SHANG Yi, YING Di, ZHAO Hui
摘要: 标题生成作为文本生成任务的一项基础应用,常面临信息覆盖不足和语义偏离的问题。针对这一挑战,提出一种以核心句子为引导的多任务标题生成模型,强调核心句子在捕捉原文主旨和提升标题生成质量中的关键作用。该模型以原文、核心句子与关键词为输入,训练阶段使用标注的核心句子,测试阶段则由核心句子分类任务自动获取。通过核心句子分类与标题生成的联合训练,模型能够在识别关键内容的同时,更精准地生成高度契合原文语义的标题。此外,为进一步提升生成效果,引入关键词与标题的相似度损失,辅助加强主题一致性。解码阶段,在教学场景中明确区分了内容理解和概念聚焦两种认知过程,模型构建了双层交叉注意力,生成概括性强且简洁流畅的标题。实验结果表明,在多任务框架下,核心句子提取任务的结果可以辅助标题生成任务,任务间通过共享信息协同优化,显著提升了标题生成的质量,为教学资源的自动化构建提供了新思路。
中图分类号:
| [1]BESNIK F,CHEN Z Y,OLEG R,et al.InstructPTS:Instruction-Tuning LLMs for Product Title Summarization[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing:Industry Track.Association for Computational Linguistics,2023:663-674. [2]ZHANG X Y,JIANG Y J,SHANG Y,et al.DSGPT:Domain-specific generative pre-training of transformers for text generation in E-commerce title and review summarization[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR'21).New York:Association for Computing Machinery,2021:2146-2150. [3]LI P,YU J,CHEN J,et al.HG-News:News Headline Generation Based on a Generative Pre-Training Model[J].IEEE Access,2021,9:107947-107957. [4]BAE S,KIM T,KIM J,et al.Summary Level Training of Sentence Rewriting for Abstractive Summarization[C]//Procee-dings of the 2nd Workshop on New Frontiers in Summarization.Association for Computational Linguistics,2019:10-20. [5]DOU Z Y,LIU P,HAYASHI H,et al.GSum:A GeneralFramework for Guided Neural Abstractive Summarization[C]//North American Chapter of the Association for Computational Linguistics.Association for Computational Linguistics,2021. [6]LIN Y,LIU P.SimCLS:A Simple Framework for Contrastive Learning of Abstractive Summarization[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 2:Short Papers).Association for Computational Linguistics,2021:1065-1072. [7]YANG W C,GU T Y,SUI R Q.A Faster Method For Generating Chinese Text Summaries-Combining Extractive Summarization And Abstractive Summarization[C]//Proceedings of the 2022 5th International Conference on Machine Learning and Natural Language Processing(MLNLP '22).Association for Computing Machinery,New York,NY,USA,2022:54-58. [8]JIAO L Y,GUO Y,LIU Y,et al.A Sequence Model for Single Document Headline Generation[J].Journal of Chinese Information Processing,2021,35(1):64-71. [9]MIHALCEA R,TARAU P.TextRank:Bringing Order intoTexts[C]//Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing(EMNLP).2004. [10]VO T.An approach of syntactical text graph representationlearning for extractive summarization[J].International Journal of Intelligent Robotics and Applications,2023,7:190-204. [11]RAKROUKI M A,ALHARBE N R,KHAYYAT M,et al.TG-SMR:A Text Summarization Algorithm Based on Topic and Graph Models[J].Computer Systems Science and Engineering,2023,45(1):395-408. [12]MALARSELVI G,PANDIAN A.Multi-layered network model for text summarization using feature representation[J].Soft Computing,2023,27(1):311-322. [13]FEIJO D D,MOREIRA V P.Improving abstractive summarization of legal rulings through textual entailment[J].Artificial Intelligence and Law,2021,31(1):91-113. [14]VO T.A novel semantic-enhanced generative adversarial net-work for abstractive text summarization[J].Soft Computing,2023,27:6267-6280. [15]ZHANG J,ZHAO Y,SALEH M,et al.PEGASUS:Pre-training with Extracted Gap-sentences for Abstractive Summarization[C]//International Conference on Machine Learning.PMLR,2020. [16]LEWIS M,LIU Y,GOYAL N,et al.BART:Denoising Se-quence-to-Sequence Pre-training for Natural Language Generation,Translation,and Comprehension[C]//58th Annual Mee-ting of the Association for Computational Linguistics.Association for Computational Linguistics,2020. [17]RAFFEL C,SHAZEER N,ROBERTS A,et al.Exploring the limits of transfer learning with aunified text-to-text transformer[J].Journal of machine learning research,2020,21(140):1-67. [18]SONG K,WANG B,FENG Z,et al.Controlling the Amount of Verbatim Copying in Abstractive Summarization[C]//National Conference on Artificial Intelligence.Association for the Advancement of Artificial Intelligence(AAAI),2020. [19]KOSUKE Y,YUTA H,HIDEAKI T,et al.Transformer-based Lexically Constrained Headline Generation[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics,2021. [20]JONES K S.Indexterm weighting[J].Information Storage &Retrieval,1973,9(11):619-633. [21]RADA M,PAUL T.TextRank:Bringing Order into Text[C]//Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics,2024. [22]DAVID M,BLE I,ANDREW Y,et al.Latent dirichlet allocation[J].The Journal of Machine Learning Research,2003,3:993-1022. [23]XIAO S Y,ZHAO H.Title generation of knowledge points for classroom teaching[J].Journal of Tsinghua University(Science and Technology),2023,64(5):770-779. [24]LEWIS M,LIU Y,GOYAL N,et al.BART:denoising sequence-to-sequence pre-training for natural language generation,translation,and comprehension [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics,2020. [25]DONG L,YANG N,WANG W H,et al.Unified language model pre-training for natural language understanding and generation[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems.Curran Associates Inc.,2019:13063-13075. [26]XUE L,CONSTANT N,ROBERTS A,et al.mT5:A massively multilingual pretrained text-to-text transformer[J].arXiv:2010.11934,2020. |
|
||