Computer Science ›› 2021, Vol. 48 ›› Issue (7): 292-298.doi: 10.11896/jsjkx.200500133

• Artificial Intelligence • Previous Articles     Next Articles

BGCN:Trigger Detection Based on BERT and Graph Convolution Network

CHENG Si-wei1, GE Wei-yi2, WANG Yu2, XU Jian1   

  1. 1 School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China
    2 Key Laboratory of Information System Engineering,28th Research Institute of China Electronic Science and Technology Group Corporation 210007,China
  • Received:2020-05-26 Revised:2020-08-10 Online:2021-07-15 Published:2021-07-02
  • About author:CHENG Si-wei,born in 1997,postgra-duate.His main research interests include natural language processing and machine learning.(735665705@qq.com)
    XU Jian,born in 1979,Ph.D, professor, master director. His main research interests include data mining and know-ledge graph.
  • Supported by:
    National Natural Science Foundation of China(61872186) and Science and Technology on Information System Engineering Laboratory(05201901).

Abstract: Trigger word detection is a basic task of event extraction,which involves the recognition and classification of trigger words.There are two main problems in the previous work:(1)the neural network model for trigger word detection only consi-ders the sequential representation of sentences,and the sequential modeling method is inefficient in capturing long-distance dependencies;(2)although the representation-based method overcomes the problem of manual feature extraction,the word vector used as the initial training feature lacks the degree of representation of the sentence,so it is difficult to capture the deep two-way representation.Therefore,we propose a trigger word detection model BGCN,based on BERT model and GCN network.This model strengthens the feature representation by introducing BERT word vector,and introduces syntactic structure to capture long-distance dependencies and detect event trigger words.Experimental results show that our method outperforms other existing neural network models on ACE2005 datasets.

Key words: BERT, Bi-LSTM, Event trigger, Graph convolution network, Sequence annotation

CLC Number: 

  • TP183
[1]GRISHMAN R,WESTBROOK D,MEYERS A.NYU’s Eng-lish ACE 2005 system description[J/OL].ACE,2005,5.http://www.researchgate.net/publication/228638184_NYU’s_English_ACE_2005_system_description.
[2]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013.
[3]PENNINGTON J,SOCHER R,MANNING C.Glove:Globalvectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing(EMNLP).2014:1532-1543.
[4]PETERS M E,NEUMANN M,IYYER M,et al.Deep contex-tualized word representations[J].arXiv:1802.05365,2018.
[5]LIU S,LIU K,HE S,et al.A probabilistic soft logic based approach to exploiting latent and global information in event classification[C]//Thirtieth AAAI Conference on Artificial Intelligence.2016.
[6]LI X,NGUYEN T H,CAO K,et al.Improving event detection with abstract meaning representation[C]//Proceedings of the First Workshop on Computing News Storylines.2015:11-15.
[7]NGUYEN T H,CHO K,GRISHMAN R.Joint event extraction via recurrent neural networks[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:300-309.
[8]CHEN Y,XU L,LIU K,et al.Event extraction via dynamicmulti-pooling convolutional neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for ComputationalLinguistics and the 7th International Joint Conference on Natu-ral Language Processing (Volume 1:Long Papers).2015:167-176.
[9]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[10]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[11]MARCHEGGIANI D,TITOV I.Encoding sentences with graph convolutional networks for semantic role labeling[J].arXiv:1703.04826,2017.
[12]NGUYEN T H,GRISHMAN R.Graph convolutional networks with argument-aware pooling for event detection[C]//Thirty-Second AAAI Conference on Artificial Intelligence.2018.
[13]LIU X,LUO Z,HUANG H.Jointly multiple events extraction via attention-based graph information aggregation[J].arXiv:1809.09078,2018.
[14]LI Q,JI H,HUANG L.Joint event extraction via structured prediction with global features[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2013:73-82.
[15]ZHANG X F,GUO Z G,LIU S,et al.Self-similarity Clustering Event Detection Based on Triggers Guid [J].Computer Science,2010,27(3):212-214.
[16]XU X,LI P F,ZHU Q M.Pattern Filtering and ConversionMethods for Semi-supervised Chinese Event Extraction.[J].Computer Science,2015,42(2):253-255.
[17]LIU S,CHENG R,YU X,et al.Exploiting contextual information via dynamic memory network for event detection[J].arXiv:1810.03449,2018.
[18]LIU S,LI Y,ZHANG F,et al.Event Detection without Triggers[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:735-744.
[19]ZHANG J,QIN Y,ZHANG Y,et al.Extracting Entities andEvents as a Single Task Using a Transition-Based Neural Model[C]//IJCAI.2019:5422-5428.
[20]ORR J W,TADEPALLI P,FERN X.Event detection with neural networks:A rigorous empirical evaluation[J].arXiv:1808.08504,2018.
[21]SHA L,QIAN F,CHANG B,et al.Jointly extracting event triggers and arguments by dependency-bridge rnn and tensor-based argument interaction[C]//Thirty-Second AAAI Conference on Artificial Intelligence.2018.
[22]LIU S,CHEN Y,LIU K,et al.Exploiting argument information to improve event detection via supervised attention mechanisms[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2017:1789-1798.
[23]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[24]YAN H R,JIN X L,MENG X B,et al.Event Detection with Multi-Order Graph Convolution and Aggregated Attention[C]//The 9th International Joint Conference on Natural Language Processing.2019:5770-5774.
[25]CUI S Y,YU B W,LIU T W,et al.Event Detection with Relation-Aware Graph Convolutional Networks[J].arXiv:2002.10757,2020.
[26]PENG H,LI J,GONG Q,et al.Fine-grained event categorization with heterogeneous graph convolutional networks[J].arXiv:1906.04580,2019.
[27]DUAN S,HE R,ZHAO W.Exploiting document level information to improve event detection via recurrent neural networks[C]//Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1:Long Papers).2017:352-361.
[28]LIU J,CHEN Y,LIU K,et al.Event detection via gated multilingual attention mechanism[C]//Thirty-Second AAAI Confe-rence on Artificial Intelligence.2018.
[1] ZHANG Yuan, KANG Le, GONG Zhao-hui, ZHANG Zhi-hong. Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM [J]. Computer Science, 2022, 49(7): 31-39.
[2] YU Jia-qi, KANG Xiao-dong, BAI Cheng-cheng, LIU Han-qing. New Text Retrieval Model of Chinese Electronic Medical Records [J]. Computer Science, 2022, 49(6A): 32-38.
[3] KANG Yan, WU Zhi-wei, KOU Yong-qi, ZHANG Lan, XIE Si-yu, LI Hao. Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution [J]. Computer Science, 2022, 49(6A): 150-158.
[4] YU Ben-gong, ZHANG Zi-wei, WANG Hui-ling. TS-AC-EWM Online Product Ranking Method Based on Multi-level Emotion and Topic Information [J]. Computer Science, 2022, 49(6A): 165-171.
[5] LI Jian-zhi, WANG Hong-ling, WANG Zhong-qing. Automatic Generation of Patent Summarization Based on Graph Convolution Network [J]. Computer Science, 2022, 49(6A): 172-177.
[6] GUO Yu-xin, CHEN Xiu-hong. Automatic Summarization Model Combining BERT Word Embedding Representation and Topic Information Enhancement [J]. Computer Science, 2022, 49(6): 313-318.
[7] XIE Yu, YANG Rui-ling, LIU Gong-xu, LI De-yu, WANG Wen-jian. Human Skeleton Action Recognition Algorithm Based on Dynamic Topological Graph [J]. Computer Science, 2022, 49(2): 62-68.
[8] DONG Zhe, SHAO Ruo-qi, CHEN Yu-liang, ZHAI Wei-feng. Named Entity Recognition in Food Field Based on BERT and Adversarial Training [J]. Computer Science, 2021, 48(5): 247-253.
[9] SHANG Xi-xue, HAN Hai-ting, ZHU Zheng-zhou. Mechanism Design of Right to Earnings of Data Utilization Based on Evolutionary Game Model [J]. Computer Science, 2021, 48(3): 144-150.
[10] WANG Wen-bo, LUO Heng-li. Complete Graph Face Clustering Based on Graph Convolution Network [J]. Computer Science, 2021, 48(11A): 275-277.
[11] CHEN De, SONG Hua-zhu, ZHANG Juan, ZHOU Hong-lin. Entity Recognition Fusing BERT and Memory Networks [J]. Computer Science, 2021, 48(10): 91-97.
[12] TIAN Wei-wei, ZHOU Yue, YIN Wang, HE Ling, DENG Li-hua and LI Yuan-yuan. Automatic Voice Detection Algorithm for Schizophrenic Combining EHHT and CI [J]. Computer Science, 2020, 47(6A): 187-195.
[13] SUN Guo-zi, LYU Jian-wei, LI Hua-kang. MeTCa:Multi-entity Trusted Confirmation Algorithm Based on Edit Distance [J]. Computer Science, 2020, 47(12): 327-331.
[14] DU Lin, CAO Dong, LIN Shu-yuan, QU Yi-qian, YE Hui. Extraction and Automatic Classification of TCM Medical Records Based on Attention Mechanism of BERT and Bi-LSTM [J]. Computer Science, 2020, 47(11A): 416-420.
[15] FENG Luan-luan, LI Jun-hui, LI Pei-feng, ZHU Qiao-ming. Technology and Terminology Detection Oriented National Defense Science [J]. Computer Science, 2019, 46(12): 231-236.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!