Computer Science ›› 2019, Vol. 46 ›› Issue (8): 244-248.doi: 10.11896/j.issn.1002-137X.2019.08.040

• Artificial Intelligence • Previous Articles     Next Articles

Event Temporal Relation Classification Method Based on Self-attention Mechanism

ZHANG Yi-jie, LI Pei-feng, ZHU Qiao-ming   

  1. (School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215006,China)
    (Province Key Lab of Computer Information Processing Technology of Jiangsu,Suzhou,Jiangsu 215006,China)
  • Received:2018-07-09 Online:2019-08-15 Published:2019-08-15

Abstract: Classifying temporal relation between events is a significant subsequent study of event extraction.With the development of deep learning,neural network plays a vital role in the task of event temporal relation classification.However,it remains a major challenge for conventional RNNs or CNNs to handle structural information and capture long distance dependence relations.To address this issue,this paper proposed a neural architecture for event temporal relation classification based on self-attention mechanism,which can directly capture relationships between two arbitrary tokens.The classification performance is improved significantly through combing this mechanism with nonlinear layers.The contrast experiments on TimeBank-Dense and Richer Event Description datasets prove that the proposed method outperforms most of the existing neural methods.

Key words: Deep learning, Self-attention mechanism, Temporal relation

CLC Number: 

  • TP391.1
[1]LIN J,YUAN C F.Extraction and Computation of Chinese Temporal Relation[J].Journal of Chinese Information Processing,2009,23(5):62-67.(in Chinese) 林静,苑春法.汉语时间关系抽取与计算[J].中文信息学报,2009,23(5):62-67.
[2]ZHONG Z M,LIU Z T,ZHOU W,et al.The Model of Event Relation Representation[J].Journal of Chinese Information Processing,2009,23(6):56-60.(in Chinese) 仲兆满,刘宗田,周文,等.事件关系表示模型[J].中文信息学报,2009,23(6):56-60.
[3]WANG F E,TAN H Y,QIAN Y L.Recognition of Temporal Relation in One Sentence Based on Maximum Entropy[J].Computer Engineering,2012,38(4):37-39.(in Chinese) 王风娥,谭红叶,钱揖丽.基于最大熵的句内时间关系识别[J].计算机工程,2012,38(4):37-39.
[4]MARCU D,ECHIHABI A.Anunsupervised approach to recognizing discourse relations[C]∥Proceedings of the Association for Computational Linguistics.Association for Computational Linguistics,2002:368-375.
[5]MANI I,VERHAGEN M,WELLNER B,et al.Machine lear- ning of temporal relations[C]∥Proceedings of the Association for Computational Linguistics.Association for Computational Linguistics,2006:753-760.
[6]CHAMBERS N,WANG S,JURAFSKY D.Classifying temporal relations between events[C]∥Proceeding of the ACL on Inte-ractive Poster and Demonstration Sessions.Association for Computational Linguistics,2007:173-176.
[7]LI P F,ZHU Q M,ZHOU G D,et al.Global Inference to Chinese Temporal Relation Extraction[C]∥Proceedings of the International Conference on Computational Linguistics.2016:1451-1460.
[8]CHENG F,MIYAO Y.Classifying Temporal Relations by Bidirectional LSTM over Dependency Paths[C]∥Proceedings of the Association for Computational Linguistics(Short Papers).Association for Computational Linguistics.2017:1-6.
[9]MENG Y,RUMSHISKY A,ROMANOV A.Temporal Infor- mation Extraction for Question Answering Using Syntactic Dependencies in an LSTM-based Architecture[C]∥Proceedings of the Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics,2017:887-896.
[10]CHOUBEY P K,HUANG R H.A Sequential Model for Classifying Temporal Relations between Intra-Sentence Events[C]∥Proceedings of the Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics.2017:1796-1802.
[11]TOURILLE J,FERRET O,TANNIER X,et al.Neural Architecture for Temporal Relation Extraction:A Bi-LSTM Approach for Detecting Narrative Containers[C]∥Proceedings of the Association for Computational Linguistics.Association for Computational Linguistics,2017:224-230.
[12]VASWANI A,SHAZEER N,PARMAR N,et al.Attentionis all you need[J].arXiv:1706.03762.
[13]CHENG J P,DONG L,LAPATA M.Long Short-Term Memory-Networks for Machine Reading[J].arXiv:1601.06733.
[14]LIN Z H,FENG M W,SANTOS C N,et al.A Structured Self-attentive Sentence Embedding[J].arXiv:1703.03130.
[15]PAULUS R,XIONG C M,SOCHER R.A Deep Reinforced Model for Abstractive Summarization[J].arXiv:1705.04304.
[16]SHEN T,ZHOU T Y,LONG G D,et al.DiSAN:Directional Self-Attention Network for RNN/CNN-free Language Understanding[J].arXiv :1709.04696.
[17]DEY R,SALEMT F M.Gate-variants of Gated Recurrent Unit (GRU)neural networks[J].arXiv:1701.05923.
[18]DAUPHIN Y N,FAN A,AULI M,et al.Language Modeling with Gated Convolutional Networks[J].arXiv:1612.08083.
[19]MIRZA P,TONELLI S.On the contribution of word embeddings to temporal relationclassification[C]∥Proceedings of the International Conference on Computational Linguistics.2016:2818-2828.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[9] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[10] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[11] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[12] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[13] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[14] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[15] ZHU Wen-tao, LAN Xian-chao, LUO Huan-lin, YUE Bing, WANG Yang. Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN [J]. Computer Science, 2022, 49(6A): 378-383.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!