基于双重注意力的无触发词中文事件检测

doi:10.11896/jsjkx.211000071

计算机科学 ›› 2023, Vol. 50 ›› Issue (1): 276-284.doi: 10.11896/jsjkx.211000071

基于双重注意力的无触发词中文事件检测

程永, 毛莺池, 万旭, 王龙宝, 朱敏

水利部水利大数据技术重点实验室南京 210098
河海大学计算机与信息学院南京 211100

收稿日期:2021-10-11 修回日期:2022-03-14 出版日期:2023-01-15 发布日期:2023-01-09
通讯作者: 毛莺池(yingchimao@hhu.edu.cn)
作者简介:yungcheng@hhu.edu.cn
基金资助:
江苏省重点研发计划(BE2020729);中国华能集团关键技术(HNKJ19-H12,HNK20-H64)

Chinese Event Detection Without Triggers Based on Dual Attention

CHENG Yong, MAO Yingchi, WAN Xu, WANG Longbao, ZHU Min

Key Laboratory of Water Big Data Technology of Ministry of Water Resources,Nanjing 210098,China
College of Computer and Information,Hohai University,Nanjing 211100,China

Received:2021-10-11 Revised:2022-03-14 Online:2023-01-15 Published:2023-01-09
About author:CHENG Yong,born in 1998,postgra-duate,is a member of China Computer Federation.His main research interests include intelligent computing and so on.
MAO Yingchi,born in 1976,Ph.D,professor,Ph.D supervisor,is a senior member of China Computer Federation.Her main research interests include edge intelligence computing and so on.
Supported by:
Key Research and Development Project of Jiangsu Province(BE2020729) and Key Technology Project of China Huaneng Group(HNKJ19-H12,HNK20-H64).

摘要/Abstract

摘要： 事件抽取是自然语言处理的重要任务,而事件检测是事件抽取的关键步骤之一,其目标是检测事件的发生并对其进行分类。目前基于触发器识别的中文事件检测方法存在一词多义、词与触发词不匹配的问题,影响了事件检测模型的精度。针对此问题,提出基于双重注意力的无触发词事件检测模型(Event Detection Without Triggers based on Dual Attention,EDWTDA),该模型可跳过触发词识别过程,实现在无触发词标记情况下直接判断事件类型。EDWTDA利用ALBERT改善词嵌入向量的语义表示能力,缓解一词多义问题,提高模型预测能力;采用局部注意力融合事件类型捕捉句中关键语义信息并模拟隐藏的事件触发词,解决词与触发词不匹配的问题;借助全局注意力挖掘文档中的语境信息,解决一词多义问题;最后将事件检测转化成二分类任务,解决多标签问题。同时,采用Focal loss损失函数解决转化成二分类后产生的样本不均衡问题。在ACE2005中文语料库上的实验结果表明,所提模型相比最佳基线模型JMCEE在精确率、召回率和F1-score评价指标上分别提高了3.40%,3.90%,3.67%。

关键词: 双重注意力, 无触发词, 中文事件检测, ACE2005, 二分类

Abstract: Event extraction is an essential task of natural language processing,and event detection is one of the critical steps of event extraction,whose goal is to detect the occurrence of events and classify them.Currently,Chinese event detection has problems of polysemous words and mismatches between words and triggers,which affect the accuracy of event detection models.We propose the event detection without triggers based on dual attention(EDWTDA),which skips the process of trigger word recognition and directly determines event types without trigger word tags.First,the ALBERT model is applied to improve the semantic representation ability of word embedding vectors.Second,we fusion local attention and event types to capture key semantic information and simulate hidden event triggers to solve the problem of mismatch between words and triggers.Third,the global attention is introduced to mine contextual information in documents to solve the problem of polysemous words.Further,the event detection task is converted into a binary classification task for solving multi-label problem.Finally,the focal loss function is used to address the sample imbalance after conversion.Experimental results on the ACE2005 Chinese corpus show that compared with the best baseline model JMCEE,the accuracy rate,recall rate,and F1-score of the proposed model increases by 3.40%,3.90% and 3.67%,respectively.

Key words: Double attention, Without triggers, Chinese event detection, ACE2005, Binary classification

中图分类号:

TP391

程永, 毛莺池, 万旭, 王龙宝, 朱敏. 基于双重注意力的无触发词中文事件检测[J]. 计算机科学, 2023, 50(1): 276-284. https://doi.org/10.11896/jsjkx.211000071

CHENG Yong, MAO Yingchi, WAN Xu, WANG Longbao, ZHU Min. Chinese Event Detection Without Triggers Based on Dual Attention[J]. Computer Science, 2023, 50(1): 276-284. https://doi.org/10.11896/jsjkx.211000071

参考文献

[1]JI H,GRISHMAN R.Refining event extraction through cross-document inference[C]//Proceedings of ACL-08:Hlt.Columbus:ACL,2008:254-262.
[2]LI Q,JI H,HUANG L.Joint event extraction via structured prediction with global features[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).Sofia:ACL,2013:73-82.
[3]CHEN Y,XU L,LIU K,et al.Event extraction via dynamic multi-pooling convolutional neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computatio-nal Linguistics and the 7th International Joint Conference on Natural Language Processing(Volume 1:Long Papers).Beijing:ACL,2015:167-176.
[4]LIU J,CHEN Y,LIU K,et al.Event detection via gated multilingual attention mechanism[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New Orleans:AAAI press,2018:4865-4872.
[5]YAN H,JIN X,MENG X,et al.Event detection with multi-order graph convolution and aggregated attention[C]//Procee-dings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).Hong Kong:ACL,2019:5770-5774.
[6]CUI S,YU B,LIU T,et al.Edge-enhanced graph convolution networks for event detection with syntactic relation[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing:Findings.2020:2329-2339.
[7]NGUYEN T H,GRISHMAN R.Graph convolutional networks with argument-aware pooling for event detection[C]//Thirty-second AAAI Conference on Artificial Intelligence.New Orleans:AAAI Press,2018:5900-5907.
[8]CAO P,CHEN Y,ZHAO J,et al.Incremental Event Detection via Knowledge Consolidation Networks[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP).2020:707-717.
[9]LIN H,LU Y,HAN X,et al.Nugget Proposal Networks forChinese Event Detection[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Vo-lume 1:Long Papers).Melbourne:ACL,2018:1565-1574.
[10]DING N,LI Z,LIU Z,et al.Event detection with trigger-aware lattice neural network[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).Hong Kong:ACL,2019:347-356.
[11]LIU S,LI Y,ZHANG F,et al.Event Detection without Triggers[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers).Minneapolis:ACL,2019:735-744.
[12]DU X,CARDIE C.Event Extraction by Answering(Almost)Natural Questions[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP).2020:671-683.
[13]NGUYEN T H,GRISHMAN R.Event detection and domain adaptation with convolutional neural networks[C]//Procee-dings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Volume 2:Short Papers).Beijing:ACL,2015:365-371.
[14]NGUYEN T H,GRISHMAN R.Modeling skip-grams for event detection with convolutional neural networks[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Austin:ACL,2016:886-891.
[15]FENG X,QIN B,LIU T.Alanguage-independent neural net-work for event detection[J].Science China Information Sciences.China:Science in China Press,2018,61(9):1-12.
[16]KODELJA D,BESANÇON R,FERRET O.Exploiting a more global context for event detection through bootstrapping[C]//European Conference on Information Retrieval.Cham:Springer,2019:763-770.
[17]JI Y,LIN Y,GAO J,et al.Exploiting the Entity Type Sequence to Benefit Event Detection[C]//Proceedings of the 23rd Confe-rence on Computational Natural Language Learning(CoNLL).Hong Kong:ACL,2019:613-623.
[18]LAI V D,NGUYEN T N,NGUYEN T H.Event detection:Gate diversity and syntactic importance scores for graph convolution neural networks[J].arXiv:2010.14123,2020.
[19]DUTTA S,MA L,SAHA T K,et al.GTN-ED:Event Detection Using Graph Transformer Networks[J].arXiv:2104.15104,2021.
[20]LIU J,CHEN Y,LIU K,et al.Neural cross-lingual event detection with minimal parallel resources[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).Hong Kong:ACL,2019:738-748.
[21]DENG S,ZHANG N,KANG J,et al.Meta-learning with dynamic-memory-based prototypical network for few-shot event detection[C]//Proceedings of the 13th International Conference on Web Search and Data Mining.Houston:Association for Computing Machinery,2020:151-159.
[22]LAI V D,DERNONCOURT F,NGUYEN T H.Extensivelymatching for few-shot learning event detection[J].arXiv:2006.10093,2020.
[23]HUANG L,JI H.Semi-supervised new event type induction and event detection[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP).2020:718-724.
[24]TONG M,XU B,WANG S,et al.Improving event detection via open-domain trigger knowledge[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.ACL,2020:5887-5897.
[25]WANG X,HAN X,LIU Z,et al.Adversarial training for weakly supervised event detection[C]//Proceedings of the 2019 Confe-rence of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers).Minneapolis:ACL,2019:998-1008.
[26]CHEN Y,YANG H,LIU K,et al.Collective event detection via a hierarchical and bias tagging networks with gated multi-level attention mechanisms[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels:ACL,2018:1267-1276.
[27]ZHAO Y,JIN X,WANG Y,et al.Document embedding en-hanced event detection with hierarchical and supervised attention[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Volume 2:Short Papers).Melbourne:ACL,2018:414-419.
[28]XIANG Y X,TONG Z,WEI Y,et al.A Hybrid Character Rep-resentation for Chinese Event Detection[C]//2019 International Joint Conference on Neural Networks(IJCNN).Budapest:IEEE,2019:1-8.
[29]ZHENG S,CAO W,XU W,et al.Doc2EDAG:An End-to-End Document-level Framework for Chinese Financial Event Extraction[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).Hong Kong:ACL,2019:337-346.
[30]BENGIO Y,DUCHARME R,VINCENT P,et al.A neuralprobabilistic language model[J].The Journal of Machine Lear-ning Research,2003,3:1137-1155.
[31]ERHAN D,COURVILLE A,BENGIO Y,et al.Why does unsupervised pre-training help deep learning? [C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics.JMLR Workshop and Conference Procee-dings.Sardinia:Microtome Publishing,2010:201-208.
[32]LAN Z,CHEN M,GOODMAN S,et al.Albert:A lite bert for self-supervised learning of language representations[J].arXiv:1909.11942,2019.
[33]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013.
[34]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pretraining of Deep Bidirectional Transformers for Language Understanding [C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers),2019:4171-4186.
[35]LAMPLE G,BALLESTEROS M,SUBRAMANIAN S,et al.Neural Architectures for Named Entity Recognition[C]//Proceedings of NAACL-HLT.San Diego:ACL,2016:260-270.
[36]XIANG W,WANG B.A survey of event extraction from text[J].IEEE Access,2019,7:173111-173137.
[37]XU N,XIE H,ZHAO D.A Novel Joint Framework for Multiple Chinese Events Extraction[C]//China National Conference on Chinese Computational Linguistics.Cham:Springer,2020:174-183.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于双重注意力的无触发词中文事件检测

Chinese Event Detection Without Triggers Based on Dual Attention

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

Metrics

本文评价

推荐阅读 0

[1]	黄扬林, 胡凯, 郭建强, 彭诚. 基于多尺度特征融合和双重注意力机制的肝脏CT图像分割 Liver CT Images Segmentation Based on Multi-scale Feature Fusion and Dual AttentionMechanism 计算机科学, 2022, 49(11A): 210800162-9. https://doi.org/10.11896/jsjkx.210800162
[2]	丁玲, 向阳. 基于分层次多粒度语义融合的中文事件检测 Chinese Event Detection with Hierarchical and Multi-granularity Semantic Fusion 计算机科学, 2021, 48(5): 202-208. https://doi.org/10.11896/jsjkx.200800038
[3]	吴凡, 朱培培, 王中卿, 李培峰, 朱巧明. 基于字词联合表示的中文事件检测方法 Chinese Event Detection with Joint Representation of Characters and Words 计算机科学, 2021, 48(4): 249-253. https://doi.org/10.11896/jsjkx.200300156
[4]	朱培培, 王中卿, 李寿山, 王红玲. 基于篇章信息和Bi-GRU的中文事件检测 Chinese Event Detection Based on Document Information and Bi-GRU 计算机科学, 2020, 47(12): 233-238. https://doi.org/10.11896/jsjkx.191100031
[5]	朱维军, 王鑫, 钟英辉, 樊永文, 陈永华. 一种基于梯度提升回归树的系外行星宜居性预测方法 Habitability Prediction of Exoplanets Based on GBRT Algorithm 计算机科学, 2019, 46(6A): 71-73.
[6]	宋瑞阳, 孟华, 龙治国. 基于数据分布特征的线性孪生支持向量机 Linear Twin Support Vector Machine Based on Data Distribution Characteristics 计算机科学, 2019, 46(6A): 407-411.