基于句间信息的图注意力卷积网络的文档级关系抽取

doi:10.11896/jsjkx.220800189

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220800189-6.doi: 10.11896/jsjkx.220800189

基于句间信息的图注意力卷积网络的文档级关系抽取

段建勇^1,2, 杨潇^1,2, 王昊^1,2,3, 何丽^1,2, 李欣^1,2

1 北方工业大学信息学院北京 100144;
2 CNONIX国家标准应用与推广实验室北京 100144;
3 北京城市治理研究基地北京 100144

出版日期:2023-06-10 发布日期:2023-06-12
通讯作者: 段建勇(duanjy@ncut.edu.cn)
基金资助:
国家自然科学基金(61972003);教育部人文社科基金项目(21YJA740052);北京哲社基地建设专项经费(110051360019XN142);北京市教育委员会科学研究计划项目(KM202210009002)

Document-level Relation Extraction of Graph Attention Convolutional Network Based onInter-sentence Information

DUAN Jianyong^1,2, YANG Xiao^1,2, WANG Hao^1,2,3, HE Li^1,2, LI Xin^1,2

1 School of Information,North China University of Technology,Beijing 100144,China;
2 CNONIX National Standard Application and Promotion Lab,Beijing 100144,China;
3 Beijing Urban Governance Research Center,Beijing 100144,China

Online:2023-06-10 Published:2023-06-12
About author:DUAN Jianyong,born in 1978,Ph.D,professor,is a member of China Computer Federation.His main research interests includd natural language processing and information retrieval.
Supported by:
National Natural Science Foundation of China(61972003),Humanities and Social Sciences Fund of the Ministry of Education(21YJA740052),Special Funds for the Construction of the Beijing Zheshe Base(110051360019XN142) and Beijing Municipal Commission of Education Scientific Research Program Project(KM202210009002).

摘要/Abstract

摘要： 为解决现有模型对文档的结构信息挖掘不足的问题,提出一种基于句间信息的图注意力卷积网络模型。该模型改进了一种文档级编码器,该编码器使用了一种新的注意力机制——句间注意力机制,使得句子的最终表示更加关注前一个句子和之前文档中的重要信息,更有利于挖掘文档的结构信息。实验结果表明,所提模型在DocRED数据集上的F₁评价指标达到56.3%,性能优于基线模型。在融入句间注意力机制时,由于模型需要对每一句话分别进行句间注意力操作,因此训练模型时需要消耗更多的内存和时间。基于句间信息的图注意力卷积网络模型可以有效地对文档中的相关信息进行聚合,并且增强对文档的结构信息的挖掘能力,从而使得模型在文档级关系抽取任务中效果得到提升。

关键词: 文档级关系抽取, 注意力机制, 文档级编码器, 图卷积网络

Abstract: In order to solve the problem of insufficient structural information mining for documents in existing models,a graph attention convolution network model based on inter-sentence information is proposed.In this model,a document level encoder is improved,which uses a new attention mechanism,inter-sentence attention mechanism,so that the final representation of a sentence pays more attention to the important information in the previous sentence and the previous document,which is more conducive to mine the structural information of the document.Experiments show that the F₁ evaluation index of this model on DocRED data set reaches 56.3%,and its performance is better than that of the baseline model.When integrating the inter sentence attention mechanism,the model needs to perform inter-sentence attention operations for each sentence,so it needs to consume more memory and time when training the model.The graph attention convolution network model based on inter-sentence information can effectively aggregate the relevant information in the document,and enhance the mining ability of the document structure information,so that the model can improve the effect of the document level relationship extraction task.

Key words: Document-level relation extraction, Attention mechanism, Document-level encoder, Graph convolutional network

中图分类号:

TP391

段建勇, 杨潇, 王昊, 何丽, 李欣. 基于句间信息的图注意力卷积网络的文档级关系抽取[J]. 计算机科学, 2023, 50(6A): 220800189-6. https://doi.org/10.11896/jsjkx.220800189

DUAN Jianyong, YANG Xiao, WANG Hao, HE Li, LI Xin. Document-level Relation Extraction of Graph Attention Convolutional Network Based onInter-sentence Information[J]. Computer Science, 2023, 50(6A): 220800189-6. https://doi.org/10.11896/jsjkx.220800189

参考文献

[1]CHE W X,LIU T,LI S.Automatic extraction of entity relationship[J].Chinese Journal of Information,2005,19(2):2-7.
[2]HUANG Q Z,ZHU S Q,FENG Y S,et al.Three sentences are all you need:Local path enhanced document relation extraction[C]//Proceedings of the 59^th Annual Meeting of the Association for Computational Linguistics and the 11^th International Joint Conference on Natural Language Processing(Volume 2:Short Papers).2021.
[3]QUIRK C,POON H.Distant supervision for relation extraction beyond the sentence boundary[C]//Proceedings of EACL.2017.
[4]KIPF T N,WELLING M.Semisupervised classification withgraph convolutional networks[C]//Proceedings of ICLR.2017.
[5]HUANG K,QI P,WANG G T,et al.Entity and evidence guided document-level relation extraction[C]//Proceedings of the 6^th Workshop on Representation Learning for NLP.2021:140-149.
[6]GUPTA P,RAJARAM S,SCHÜTZE H,et al.Neural relation extraction within and across sentence boundaries[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2019:6513-6520.
[7]NAN G,GUO Z,SEKULIC I,et al.Reasoning with LatentStructure Refinement for Document-Level Relation Extraction[C]//ACL.2020:1073-1087.
[8]ZHOU W X,HUANG K,MA T Y,et al.Document-level relation extraction with adaptive thresholding and localized context pooling[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:1631-1640.
[9]YAO Y,YE D M,LI P,et al.DocRED:A large-scale document-level relation extraction dataset[C]//Proceedings of ACL.2019:189-198.
[10]TODERICI G,O’MALLEY S M,HWANG S J,et al.Recurrent Neural Network Regularization[J].arXiv:1409.2329,2015.
[11]ZHOU P,SHI W,TIAN J,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of ACL.2016:1412-1421.
[12]DEVLIN J,WEI M,CHANG K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[J].arXiv:1810.04805v2,2019.
[13]VASWANI A,SHAZEER N,PARMAR N.Attention Is All You Need[J].arXiv:1706.03762v5,2017.
[14]GU J H,SUN F Q,QIAN L H,et al.Database:The Journal of Biological Databases and Curation[J].The Journal of Biological Databases and Curation,2017:2049-2058.
[15]CHUNG J Y,GULCEHRE C,CHO K H.Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling[J].arXiv:1412.3555,2014.
[16]GUO Z J,ZHANG Y,LU W.Attention Guided Graph Convolutional Networks for Relation Extraction[C]//Proceedings of ACL.2019:241-251.
[17]CHRISTOPOULOU F,MIWA M,ANANIADOU S.Connec-ting the dots:Document-level neural relation extraction with edge-oriented graphs[C]//Proceedings of EMNLP.2019:1532-1543.
[18]WU T,KONG F.Document level relation extraction based on graph attention convolution neural network[J].Chinese Journal of information,2021,35(10):73-80.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于句间信息的图注意力卷积网络的文档级关系抽取

Document-level Relation Extraction of Graph Attention Convolutional Network Based onInter-sentence Information

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

Metrics

本文评价

推荐阅读 0