计算机科学 ›› 2024, Vol. 51 ›› Issue (12): 234-241.doi: 10.11896/jsjkx.231100023

• 人工智能 • 上一篇    下一篇

DE-AA:基于词对距离嵌入和轴向注意力机制的实体关系联合抽取模型

张梦赢, 沈海龙   

  1. 东北大学理学院 沈阳 110819
  • 收稿日期:2023-11-02 修回日期:2024-03-28 出版日期:2024-12-15 发布日期:2024-12-10
  • 通讯作者: 沈海龙(hailong_shen@126.com)
  • 作者简介:(zhangmengying0316@163.com)

Joint Extraction of Entities and Relations Based on Word-Pair Distance Embedding and Axial Attention Mechanism

ZHANG Mengying, SHEN Hailong   

  1. School of Science, Northeastern University, Shenyang 110819, China
  • Received:2023-11-02 Revised:2024-03-28 Online:2024-12-15 Published:2024-12-10
  • About author:ZHANG Mengying,born in 2000,postgraduate.Her main research interests include knowledge graph and information extraction.
    SHEN Hailong,born in 1971,Ph.D,associate professor.His main research interests include data analysis and intelligent computing.

摘要: 实体关系联合抽取为知识图谱的构建提供了关键的技术支持,而重叠关系问题一直都是联合抽取模型研究的重点。现有的方法大多采用多步骤的建模方法,虽然在解决重叠关系问题上取得了很好的效果,但产生了曝光偏差问题。为同时解决重叠关系和曝光偏差问题,提出了一种基于词对距离嵌入和轴向注意力机制的实体关系联合抽取方法(DE-AA)。首先,构建代表词对关系的表特征,加入词对距离特征信息优化其表示;其次,应用基于行注意力和列注意力的轴向注意力模型去增强表特征,在融合全局特征的同时能够降低计算复杂度;最后,将表特征映射到各关系空间中,生成特定关系下的词对关系表,并使用表格填充法为表中各项分配标签,以三重分类的方式进行三元组的抽取。在公开数据集NYT和WebNLG上评估了所提出的模型,实验结果表明其与其他基线模型相比取得了更好的性能,且在处理重叠关系或多重关系问题上优势显著。

关键词: 实体关系联合抽取, 轴向注意力机制, 词对距离嵌入, 表格填充法

Abstract: The joint extraction of entities and relations provides key technical support for the construction of knowledge graphs,and the problem of overlapping relations has always been the focus of joint extraction model research.Many of the existing me-thods use multi-step modeling methods.Although they have achieved good results in solving the problem of overlapping relations,they have produced the problem of exposure bias.In order to solve the problem of overlapping relations and exposure bias at the same time,a joint entities and relations extraction method(DE-AA) based on word-pair distance embedding and axial attention mechanism is proposed.Firstly,the table features of the representative word-pair relation are constructed,and the word-pair distance feature information is added to optimize its representation.Secondly,the axial attention model based on row attention and column attention is applied to enhance the table features,which can reduce the computational complexity while fusing the global features.Finally,the table features are mapped to each relation space to generate the relation-specific word-pair relation table,and the table filling method is used to assign labels to each item in the table,and the triples are extracted by triple classification.The proposed model is evaluated on the public datasets NYT and WebNLG.Experimental results show that the proposed model achieves better performance than other baseline models,and has significant advantages in dealing with overlapping relations or multiple relations.

Key words: Joint extraction of entities and relations, Axial attention mechanism, Word-Pair distance embedding, Table filling method

中图分类号: 

  • TP391.1
[1]HOGAN A,BLOMQVIST E,COCHEZ M,et al.Knowledgegraphs[J].ACM Computing Surveys,2021,54(4):1-37.
[2]NASAR Z,JAFFRY S W,MALIK M K.Named entity recognition and relation extraction:State-of-the-art[J].ACM Computing Surveys,2021,54(1):1-39.
[3]HAO Y,ZHANG Y,LIU K,et al.An end-to-end model forquestion answering over knowledge base with cross-attention combining global knowledge[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2017:221-231.
[4]WANG X,HE X,CAO Y,et al.Kgat:Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.2019:950-958.
[5]LI J,SUN A,HAN J,et al.A survey on deep learning for named entity recognition[J].IEEE Transactions on Knowledge and Data Engineering,2020,34(1):50-70.
[6]KAMBAR M E Z N,ESMAEILZADEH A,HEIDARI M.Asurvey on deep learning techniques for joint named entities and relation extraction[C]//2022 IEEE World AI IoT Congress(AIIoT).IEEE,2022:218-224.
[7]ZHANG X W,WANG X,CHEN Z R,et al.Survey of Supervised Joint Entity Relation Extraction Methods[J].Journal of Frontiers of Computer Science & Technology,2022,16(4):713-733.
[8]FU T J,LI P H,MA W Y.Graphrel:Modeling text as relationalgraphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computa-tional Linguistics.2019:1409-1418.
[9]YAN Z,ZHANG C,FU J,et al.A partition filter network for joint entity and relation extraction[J].arXiv:2108.12202,2021.
[10]WEI Z,SU J,WANG Y,et al.A novel cascade binary taggingframework for relational triple extraction[J].arXiv:1909.03227,2019.
[11]YUAN Y,ZHOU X,PAN S,et al.A relation-specific attention network for joint entity and relation extraction[C]//International joint Conference on Artificial Intelligence.2021.
[12]WANG Y,YU B,ZHANG Y,et al.TPLinker:Single-stage joint extraction of entities and relations through token pair linking[J].arXiv:2010.13415,2020.
[13]REN F,ZHANG L,ZHAO X,et al.A simple but effective bidirectional framework for relational triple extraction[C]//Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining.2022:824-832.
[14]REN F,ZHANG L,YIN S,et al.A Conditional Cascade Model for Relational Triple Extraction[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management.2021:3393-3397.
[15]GOYAL A,GUPTA V,KUMAR M.Recent named entity re-cognition and classification techniques:a systematic review[J].Computer Science Review,2018,29:21-43.
[16]LIU X,CHEN H,XIA W.Overview of named entity recognition[J].Journal of Contemporary Educational Research,2022,6(5):65-68.
[17]LI Q,JI H.Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2014:402-412.
[18]ZHAO K,XU H,CHENG Y,et al.Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction[J].Knowledge-Based Systems,2021,219:106888.
[19]MIWA M,BANSAL M.End-to-end relation extraction usinglstms on sequences and tree structures[J].arXiv:1601.00770,2016.
[20]ZHENG S,WANG F,BAO H,et al.Joint extraction of entities and relations based on a novel tagging scheme[J].arXiv:1706.05075,2017.
[21]WANG J,LU W.Two are better than one:Joint entity and relation extraction with table-sequence encoders[J].arXiv:2010.03851,2020.
[22]DAI D,XIAO X,LYU Y,et al.Joint extraction of entities andoverlapping relations using position-attentive sequence labeling[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:6300-6308.
[23]ZENG X,ZENG D,HE S,et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computa-tional Linguistics(Volume 1:Long Papers).2018:506-514.
[24]EBERTS M,ULGES A.Span-based joint entity and relation extraction with transformer pre-training[J].arXiv:1909.07755,2019.
[25]ZHENG H,WEN R,CHEN X,et al.PRGC:Potential relation and global correspondence based joint relational triple extraction[J].arXiv:2106.09895,2021.
[26]REN F,ZHANG L,YIN S,et al.A novel global feature-oriented relational triple extraction model based on table filling[J].ar-Xiv:2109.06705,2021.
[27]HO J,KALCHBRENNER N,WEISSENBORN D,et al.Axial attention in multidimensional transformers[J].arXiv:1912.12180,2019.
[28]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[29]SHANG Y M,HUANG H,MAO X.Onerel:Joint entity and relation extraction with one module in one step[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:11285-11293.
[30]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[31]RIEDEL S,YAO L,MCCALLUM A.Modeling relations andtheir mentions without labeled text[C]//Machine Learning and Knowledge Discovery in Databases:European Conference,ECML PKDD 2010,Barcelona,Spain,September 20-24,2010,Proceedings,Part III 21.Springer Berlin Heidelberg,2010:148-163.
[32]GARDENT C,SHIMORINA A,NARAYAN S,et al.Creatingtraining corpora for nlg micro-planning[C]//55th annual mee-ting of the Association for Computational Linguistics(ACL).2017.
[33]YU B,ZHANG Z,SHU X,et al.Joint extraction of entities and relations based on a novel decomposition strategy[J].arXiv:1909.04273,2019.
[34]XU B,WANG Q,LYU Y,et al.EmRel:Joint Representation of Entities and Embedded Relations for Multi-triple Extraction[C]//Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2022:659-665.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!