Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (6A): 220800189-6.doi: 10.11896/jsjkx.220800189

• Artificial Intelligence • Previous Articles     Next Articles

Document-level Relation Extraction of Graph Attention Convolutional Network Based onInter-sentence Information

DUAN Jianyong1,2, YANG Xiao1,2, WANG Hao1,2,3, HE Li1,2, LI Xin1,2   

  1. 1 School of Information,North China University of Technology,Beijing 100144,China;
    2 CNONIX National Standard Application and Promotion Lab,Beijing 100144,China;
    3 Beijing Urban Governance Research Center,Beijing 100144,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:DUAN Jianyong,born in 1978,Ph.D,professor,is a member of China Computer Federation.His main research interests includd natural language processing and information retrieval.
  • Supported by:
    National Natural Science Foundation of China(61972003),Humanities and Social Sciences Fund of the Ministry of Education(21YJA740052),Special Funds for the Construction of the Beijing Zheshe Base(110051360019XN142) and Beijing Municipal Commission of Education Scientific Research Program Project(KM202210009002).

PDF (PC)

321

Abstract: In order to solve the problem of insufficient structural information mining for documents in existing models,a graph attention convolution network model based on inter-sentence information is proposed.In this model,a document level encoder is improved,which uses a new attention mechanism,inter-sentence attention mechanism,so that the final representation of a sentence pays more attention to the important information in the previous sentence and the previous document,which is more conducive to mine the structural information of the document.Experiments show that the F1 evaluation index of this model on DocRED data set reaches 56.3%,and its performance is better than that of the baseline model.When integrating the inter sentence attention mechanism,the model needs to perform inter-sentence attention operations for each sentence,so it needs to consume more memory and time when training the model.The graph attention convolution network model based on inter-sentence information can effectively aggregate the relevant information in the document,and enhance the mining ability of the document structure information,so that the model can improve the effect of the document level relationship extraction task.

Key words: Document-level relation extraction, Attention mechanism, Document-level encoder, Graph convolutional network

CLC Number: 

  • TP391
[1]CHE W X,LIU T,LI S.Automatic extraction of entity relationship[J].Chinese Journal of Information,2005,19(2):2-7.
[2]HUANG Q Z,ZHU S Q,FENG Y S,et al.Three sentences are all you need:Local path enhanced document relation extraction[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 2:Short Papers).2021.
[3]QUIRK C,POON H.Distant supervision for relation extraction beyond the sentence boundary[C]//Proceedings of EACL.2017.
[4]KIPF T N,WELLING M.Semisupervised classification withgraph convolutional networks[C]//Proceedings of ICLR.2017.
[5]HUANG K,QI P,WANG G T,et al.Entity and evidence guided document-level relation extraction[C]//Proceedings of the 6th Workshop on Representation Learning for NLP.2021:140-149.
[6]GUPTA P,RAJARAM S,SCHÜTZE H,et al.Neural relation extraction within and across sentence boundaries[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2019:6513-6520.
[7]NAN G,GUO Z,SEKULIC I,et al.Reasoning with LatentStructure Refinement for Document-Level Relation Extraction[C]//ACL.2020:1073-1087.
[8]ZHOU W X,HUANG K,MA T Y,et al.Document-level relation extraction with adaptive thresholding and localized context pooling[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:1631-1640.
[9]YAO Y,YE D M,LI P,et al.DocRED:A large-scale document-level relation extraction dataset[C]//Proceedings of ACL.2019:189-198.
[10]TODERICI G,O’MALLEY S M,HWANG S J,et al.Recurrent Neural Network Regularization[J].arXiv:1409.2329,2015.
[11]ZHOU P,SHI W,TIAN J,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of ACL.2016:1412-1421.
[12]DEVLIN J,WEI M,CHANG K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[J].arXiv:1810.04805v2,2019.
[13]VASWANI A,SHAZEER N,PARMAR N.Attention Is All You Need[J].arXiv:1706.03762v5,2017.
[14]GU J H,SUN F Q,QIAN L H,et al.Database:The Journal of Biological Databases and Curation[J].The Journal of Biological Databases and Curation,2017:2049-2058.
[15]CHUNG J Y,GULCEHRE C,CHO K H.Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling[J].arXiv:1412.3555,2014.
[16]GUO Z J,ZHANG Y,LU W.Attention Guided Graph Convolutional Networks for Relation Extraction[C]//Proceedings of ACL.2019:241-251.
[17]CHRISTOPOULOU F,MIWA M,ANANIADOU S.Connec-ting the dots:Document-level neural relation extraction with edge-oriented graphs[C]//Proceedings of EMNLP.2019:1532-1543.
[18]WU T,KONG F.Document level relation extraction based on graph attention convolution neural network[J].Chinese Journal of information,2021,35(10):73-80.
[1] ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[2] TENG Sihang, WANG Lie, LI Ya. Non-autoregressive Transformer Chinese Speech Recognition Incorporating Pronunciation- Character Representation Conversion [J]. Computer Science, 2023, 50(8): 111-117.
[3] WANG Jiahao, ZHONG Xin, LI Wenxiong, ZHAO Dexin. Human Activity Recognition with Meta-learning and Attention [J]. Computer Science, 2023, 50(8): 193-201.
[4] WANG Yu, WANG Zuchao, PAN Rui. Survey of DGA Domain Name Detection Based on Character Feature [J]. Computer Science, 2023, 50(8): 251-259.
[5] YAN Mingqiang, YU Pengfei, LI Haiyan, LI Hongsong. Arbitrary Image Style Transfer with Consistent Semantic Style [J]. Computer Science, 2023, 50(7): 129-136.
[6] GAO Xiang, TANG Jiqiang, ZHU Junwu, LIANG Mingxuan, LI Yang. Study on Named Entity Recognition Method Based on Knowledge Graph Enhancement [J]. Computer Science, 2023, 50(6A): 220700153-6.
[7] ZHANG Tao, CHENG Yifei, SUN Xinxu. Graph Attention Networks Based on Causal Inference [J]. Computer Science, 2023, 50(6A): 220600230-9.
[8] CUI Lin, CUI Chenlu, LIU Zhengwei, XUE Kai. Speech Emotion Recognition Based on Improved MFCC and Parallel Hybrid Model [J]. Computer Science, 2023, 50(6A): 220800211-7.
[9] YANG Xing, SONG Lingling, WANG Shihui. Remote Sensing Image Classification Based on Improved ResNeXt Network Structure [J]. Computer Science, 2023, 50(6A): 220100158-6.
[10] ZHANG Shunyao, LI Huawang, ZHANG Yonghe, WANG Xinyu, DING Guopeng. Image Retrieval Based on Independent Attention Mechanism [J]. Computer Science, 2023, 50(6A): 220300092-6.
[11] LIU Haowei, YAO Jingchi, LIU Bo, BI Xiuli, XIAO Bin. Two-stage Method for Restoration of Heritage Images Based on Muti-scale Attention Mechanism [J]. Computer Science, 2023, 50(6A): 220600129-8.
[12] LI Fan, JIA Dongli, YAO Yumin, TU Jun. Graph Neural Network Few Shot Image Classification Network Based on Residual and Self-attention Mechanism [J]. Computer Science, 2023, 50(6A): 220500104-5.
[13] SUN Kaiwei, WANG Zhihao, LIU Hu, RAN Xue. Maximum Overlap Single Target Tracking Algorithm Based on Attention Mechanism [J]. Computer Science, 2023, 50(6A): 220400023-5.
[14] WU Liuchen, ZHANG Hui, LIU Jiaxuan, ZHAO Chenyang. Defect Detection of Transmission Line Bolt Based on Region Attention Mechanism andMulti-scale Feature Fusion [J]. Computer Science, 2023, 50(6A): 220200096-7.
[15] ZHANG Shuaiyu, PENG Li, DAI Feifei. Person Re-identification Method Based on Progressive Attention Pyramid [J]. Computer Science, 2023, 50(6A): 220200084-8.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.