计算机科学 ›› 2023, Vol. 50 ›› Issue (2): 275-284.doi: 10.11896/jsjkx.220400271

• 人工智能 • 上一篇    下一篇

基于会话式机器阅读理解模型的事件抽取方法

刘露平1, 周欣1,2, 程军军2, 何小海1, 卿粼波1, 王美玲1   

  1. 1 四川大学电子信息学院 成都 610065
    2 中国信息安全测评中心 北京 100085
  • 收稿日期:2022-04-27 修回日期:2022-10-26 出版日期:2023-02-15 发布日期:2023-02-22
  • 通讯作者: 何小海(hxh@scu.edu.cn)
  • 作者简介:(lupingllp@gmail.com)
  • 基金资助:
    国家自然科学基金(60903098);成都市重大应用示范项目(2019-YF09-00120-SN)

Event Extraction Method Based on Conversational Machine Reading Comprehension Model

LIU Luping1, ZHOU Xin1,2, CHEN Junjun2, He Xiaohai1, QING Linbo1, WANG Meiling1   

  1. 1 College of Electronics and Information Engineering,Sichuan University,Chengdu 610065,China
    2 China Information Technology Security Evaluation Center,Beijing,100085,China
  • Received:2022-04-27 Revised:2022-10-26 Online:2023-02-15 Published:2023-02-22
  • Supported by:
    National Natural Science Foundation of China(60903098) and Chengdu Major Technology Application Demonstration Project(2019-YF09-00120-SN)

摘要: 事件抽取旨在从海量的非结构化文本中自动提取出结构化描述信息,以帮助人们快速地了解事件的最新发展动态。传统的事件抽取方法主要采用分类或者序列标注的方法,其依赖于大量的标注数据来训练模型。近年来,研究者提出了利用机器阅读理解模型来进行事件抽取的方法,通过任务转换并联合利用机器阅读理解任务中的标注数据进行训练来缓解标注数据的不足。然而现有方法局限于单轮问答,问答对之间缺少依赖关系;此外,已有方法也未充分利用句子中的实体信息等知识。针对以上不足,提出了一种会话式机器阅读理解框架用于事件抽取,针对已有方法进行了两方面的扩展:首先,通过在句子中显式地增加实体标记信息,使得模型能够有效地学习到输入句子中的实体知识;其次,设计了历史会话信息编码模块,并结合注意力机制从历史会话中筛选出重要信息,融合到阅读理解模型中以辅助推断。最后,在公开数据集上的实验结果表明所提模型相比已有方法取得了更优的结果。

关键词: 事件抽取, 会话式机器阅读理解, 实体信息标记, 历史会话信息编码, 注意力机制

Abstract: Event extraction aims to extract structured information automatically from massive unstructured texts to help people quickly understand the latest developments of events.Traditional methods are mainly implemented by classification or sequence labeling methods,which rely on a large amount of labeled data to train the model.In recent years,researchers have proposed to use machinereading comprehension models for event extraction,and through task conversion and combined use of machine rea-ding comprehension datasets for training to effectively alleviate the issue of insufficient annotation data.However,existing methods are limited to a single round of question answering and lack dependencies between different question and answer rounds.In addition,existing methods do not fully utilize entity knowledge in sentences.To this end,a new machine reading comprehension model for event extraction is proposed,and we extend existing methods in two ways.Firstly,by explicitly adding entity tag information in the sentence,making the model effectively learn the prior knowledge of the entities in the input sentence.Secondly,a historical conversational information encoding module is designed,and the attention mechanism is utilized to select important information from historical conversations to assist in inference.Finally,experiment results on a public dataset show that the new model achieves better performance than the existing methods based on the machine reading comprehension model.

Key words: Event extraction, Conversational machine reading comprehension, Entity information marking, Historical conversational information encoding, Attention mechanism

中图分类号: 

  • TP391
[1]HE R F,DUAN S Y.Joint Chinese event extraction basedmulti-task learning[J].Journal of Software,2019,30(4):1015-1030.
[2]LI W,WU M,LU Q,et al.Extractive summarization using inter-and intra-event relevance[C]//Proceeding of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2006:369-376.
[3]HAO Y Z,ZHENG Q H,CHEN Y P,et al.Recognition of ab-normal behavior based on data of public opinion on the web[J].Journal of Computer Research and Development,2016,53(3):611-620.
[4]YANG H,CHEN Y,LIU K,et al.DCFEE:A document-levelChinese financial event extraction system based on automatically labeled training data[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2018:50-55.
[5]XUE C,GAO N,ZHA D R,et al.Event database construction techniques[J].Journal of Cyber Security,2019,4(2):83-106.
[6]BAI L,ZHOU Z Y,LI B Y,et al.The construction of the even-tic graph for the political field[J].Journal of Chinese Information Processing,2021,35(4):66-74,82.
[7]GAROFOLO J.Automatic content extraction(ACE) [EB/OL].Philadelphia,PA:University of Pennsylvanial.(2005-07-01) [2021-10-03].http://itl.gov/iad/mig/-tests/ace/2005.
[8]LIU K,CHEN Y,LIU J,et al.Extracting event and their relations from texts:a survey on recent research progress and challenges[J].AI Open,2020,1:22-39.
[9]LLORENS H,SAQUETE E,NAVARRO B.TimeML eventsrecognition and classification:learning CRF models with semantic roles[C]//Proceedings of the 23rd International Conference on Computational Linguistics.Stroudsburg,PA:ACL,2010:725-733.
[10]AHN D.Thestages of event extraction[C]//Proceedings of the Workshop on Annotating and Reasoning about Time and Events.Stroudsburg,PA:ACL,2006:1-8.
[11]HONG Y,ZHANG J,MA B,et al.Using cross-entity inference to improve event extraction[C]//Proceedings of the 49th An-nual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2011:1127-1136.
[12]LIAO S,GRISHMAN R.Using document level cross-event inference to improve event extraction[C]//Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2010:789-797.
[13]NGUYEN T H,GRISHMAN R.Event detection and domainadaptation with convolutional neural networks[C]//Procee-dings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Int Joint Conf on Natural Language Processing.Stroudsburg:ACL,2015:365-371.
[14]CHEN Y,XU L,LIU K,et al.Event extraction via dynamic multi-pooling convolutional neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computa-tional Linguistics and the 7th Int Joint Conf on Natural Language Processing.Stroudsburg:ACL,2015:167-176.
[15]FENG X,HUANG L,TANG D,et al.A language-independent neural network for event detection[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2016:66-71.
[16]SHA L,QIAN F,CHANG B,et al.Jointly extracting event triggers and arguments by dependency-bridge rnn and tensor-based argument interaction[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park,CA:AAAI,2018:5916-5923.
[17]YANG S,FENG D,QIAO L,et al.Exploring pre-trained lan-guage models for event extraction and generation[C]//Procee-dings of the 57th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2019:5284-5294.
[18]DENG S,ZHANG N,LI L,et al.OntoED:Low-resource Event Detection with Ontology Embedding[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing.Stroudsburg:ACL,2021:2828-2839.
[19]LYU Q,ZHANG H,SULEM E,et al.Zero-shot event extraction via transfer learning:Challenges and insights[C]//Procee-dings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 2:Short Papers).2021:322-332.
[20]CONG X,CUI S,YU B,et al.Few-Shot Event Detection with Prototypical Amortized Conditional Random Field[C]//Proceedings of the Findings of the Association for Computational Linguistics.Stroudsburg:ACL,2021:28-40.
[21]DU X,CARDIE C.Event extraction by answering(almost) natural questions[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:ACL,2020:671-683.
[22]LIU J,CHEN Y,LIU K,et al.Event extraction as machinereading comprehension[C]//Proceedings of the 2020 Confe-rence on Empirical Methods in Natural Language Processing.Stroudsburg:ACL,2020:1641-1651.
[23]LI F,PENG W,CHEN Y,et al.Event extraction as multi-turn question answering[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing:Findings.Stroudsburg:ACL,2020:829-838.
[24]LI Q,PENG H,LI J,et al.Reinforcement learning-based dialogue guided event extraction to exploit argument relations[J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2021,30:520-533.
[25]DEVLIN J,CHANG M W,LEE K,et al.BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics.Stroudsburg,PA:ACL,2019:4171-4186.
[26]GU Y J,GUI X L,LI D F,et al.Survey of machine reading comprehension based on neural network[J].Journal of Software,2020,31(7):2095-2126.
[27]SOARES L B,FITZGERALD N,LING J,et al.Matching the blanks:distributional similarity for relation learning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2019:2895-2905.
[28]PASZKE A,GROSS S,MASSA F,et al.PyTorch:an imperative style,high-performance deep learning library[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems.Cambridge,MA:MIT Press,2019:8026-8037.
[29]KINGMA D P,BA J.Adam:A method for stochastic optimization [EB/OL].(2014-12-22) [2021-08-15].https://arxiv.org/abs/1412.6980.
[30]RAJPURKAR P,JIA R,LIANG P.Know what you don'tknow:unanswerable questions for squad[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2018:784-789.
[31]LI Q,JI H,HUANG L.Joint event extraction via structured prediction with global features[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2013:73-82.
[32]NGUYEN T H,CHO K,GRISHMAN R.Joint event extraction via recurrent neural networks[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics.Stroudsburg,PA:ACL,2016:300-309.
[33]LIU X,LUO Z,HUANG H Y.Jointly multiple events extractions via attention-based graphinformation aggregation[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,ACL,2018:1247-1256.
[34]NGUYEN T M,NGUYEN T H.One for all:neural joint mode-ling of entities and events[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.Menlo Park,CA:AAAI,2019:6851-6858.
[35]ZHANG T,JI H,SIL A.Joint entity and event extraction with generative adversarial imitation learning[J].Data Intelligence,2019,1(2):99-120.
[1] 李帅, 徐彬, 韩祎珂, 廖同鑫.
SS-GCN:情感增强和句法增强的方面级情感分析模型
SS-GCN:Aspect-based Sentiment Analysis Model with Affective Enhancement and Syntactic Enhancement
计算机科学, 2023, 50(3): 3-11. https://doi.org/10.11896/jsjkx.220700238
[2] 陈富强, 寇嘉敏, 苏利敏, 李克.
基于图神经网络的多信息优化实体对齐模型
Multi-information Optimized Entity Alignment Model Based on Graph Neural Network
计算机科学, 2023, 50(3): 34-41. https://doi.org/10.11896/jsjkx.220700242
[3] 周明强, 代开浪, 吴全旺, 朱庆生.
异构信息网络的注意力感知多通道图卷积评分预测模型
Attention-aware Multi-channel Graph Convolutional Rating Prediction Model for Heterogeneous Information Networks
计算机科学, 2023, 50(3): 129-138. https://doi.org/10.11896/jsjkx.220300004
[4] 冯程程, 刘派, 姜琳颖, 梅笑寒, 郭贵冰.
文档增强型知识库问答
Document-enhanced Question Answering over Knowledge-Bases
计算机科学, 2023, 50(3): 266-275. https://doi.org/10.11896/jsjkx.220300022
[5] 邹芸竹, 杜圣东, 滕飞, 李天瑞.
一种基于多模态深度特征融合的视觉问答模型
Visual Question Answering Model Based on Multi-modal Deep Feature Fusion
计算机科学, 2023, 50(2): 123-129. https://doi.org/10.11896/jsjkx.211200303
[6] 王鹏宇, 台文鑫, 刘芳, 钟婷, 罗绪成, 周帆.
基于数据增强的自监督飞行航迹预测
Self-supervised Flight Trajectory Prediction Based on Data Augmentation
计算机科学, 2023, 50(2): 130-137. https://doi.org/10.11896/jsjkx.211200016
[7] 瞿中, 王彩云.
基于注意力机制和轻量级空洞卷积的混凝土路面裂缝检测
Crack Detection of Concrete Pavement Based on Attention Mechanism and Lightweight DilatedConvolution
计算机科学, 2023, 50(2): 231-236. https://doi.org/10.11896/jsjkx.211200290
[8] 蔡肖, 陈志华, 盛斌.
基于移位窗口金字塔Transformer的遥感图像目标检测
SPT:Swin Pyramid Transformer for Object Detection of Remote Sensing
计算机科学, 2023, 50(1): 105-113. https://doi.org/10.11896/jsjkx.211100208
[9] 张婧媛, 王宏霞, 何沛松.
基于Transformer的多任务图像拼接篡改检测算法
Multitask Transformer-based Network for Image Splicing Manipulation Detection
计算机科学, 2023, 50(1): 114-122. https://doi.org/10.11896/jsjkx.211100269
[10] 李雪辉, 张拥军, 史殿习, 徐化池, 史燕燕.
融合注意力特征的无锚框视觉目标跟踪方法
AFTM:Anchor-free Object Tracking Method with Attention Features
计算机科学, 2023, 50(1): 138-146. https://doi.org/10.11896/jsjkx.211000083
[11] 赵倩, 周冬明, 杨浩, 王长城.
残差注意力与多特征融合的图像去模糊
Image Deblurring Based on Residual Attention and Multi-feature Fusion
计算机科学, 2023, 50(1): 147-155. https://doi.org/10.11896/jsjkx.211100161
[12] 郑诚, 梅亮, 赵伊研, 张苏航.
基于双向注意力机制和门控图卷积网络的文本分类方法
Text Classification Method Based on Bidirectional Attention and Gated Graph Convolutional Networks
计算机科学, 2023, 50(1): 221-228. https://doi.org/10.11896/jsjkx.211100095
[13] 周芳泉, 成卫青.
基于全局增强图神经网络的序列推荐
Sequence Recommendation Based on Global Enhanced Graph Neural Network
计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[14] 戴禹, 许林峰.
基于文本行匹配的跨图文本阅读方法
Cross-image Text Reading Method Based on Text Line Matching
计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032
[15] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!