Computer Science ›› 2024, Vol. 51 ›› Issue (12): 234-241.doi: 10.11896/jsjkx.231100023

• Artificial Intelligence • Previous Articles     Next Articles

Joint Extraction of Entities and Relations Based on Word-Pair Distance Embedding and Axial Attention Mechanism

ZHANG Mengying, SHEN Hailong   

  1. School of Science, Northeastern University, Shenyang 110819, China
  • Received:2023-11-02 Revised:2024-03-28 Online:2024-12-15 Published:2024-12-10
  • About author:ZHANG Mengying,born in 2000,postgraduate.Her main research interests include knowledge graph and information extraction.
    SHEN Hailong,born in 1971,Ph.D,associate professor.His main research interests include data analysis and intelligent computing.

Abstract: The joint extraction of entities and relations provides key technical support for the construction of knowledge graphs,and the problem of overlapping relations has always been the focus of joint extraction model research.Many of the existing me-thods use multi-step modeling methods.Although they have achieved good results in solving the problem of overlapping relations,they have produced the problem of exposure bias.In order to solve the problem of overlapping relations and exposure bias at the same time,a joint entities and relations extraction method(DE-AA) based on word-pair distance embedding and axial attention mechanism is proposed.Firstly,the table features of the representative word-pair relation are constructed,and the word-pair distance feature information is added to optimize its representation.Secondly,the axial attention model based on row attention and column attention is applied to enhance the table features,which can reduce the computational complexity while fusing the global features.Finally,the table features are mapped to each relation space to generate the relation-specific word-pair relation table,and the table filling method is used to assign labels to each item in the table,and the triples are extracted by triple classification.The proposed model is evaluated on the public datasets NYT and WebNLG.Experimental results show that the proposed model achieves better performance than other baseline models,and has significant advantages in dealing with overlapping relations or multiple relations.

Key words: Joint extraction of entities and relations, Axial attention mechanism, Word-Pair distance embedding, Table filling method

CLC Number: 

  • TP391.1
[1]HOGAN A,BLOMQVIST E,COCHEZ M,et al.Knowledgegraphs[J].ACM Computing Surveys,2021,54(4):1-37.
[2]NASAR Z,JAFFRY S W,MALIK M K.Named entity recognition and relation extraction:State-of-the-art[J].ACM Computing Surveys,2021,54(1):1-39.
[3]HAO Y,ZHANG Y,LIU K,et al.An end-to-end model forquestion answering over knowledge base with cross-attention combining global knowledge[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2017:221-231.
[4]WANG X,HE X,CAO Y,et al.Kgat:Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.2019:950-958.
[5]LI J,SUN A,HAN J,et al.A survey on deep learning for named entity recognition[J].IEEE Transactions on Knowledge and Data Engineering,2020,34(1):50-70.
[6]KAMBAR M E Z N,ESMAEILZADEH A,HEIDARI M.Asurvey on deep learning techniques for joint named entities and relation extraction[C]//2022 IEEE World AI IoT Congress(AIIoT).IEEE,2022:218-224.
[7]ZHANG X W,WANG X,CHEN Z R,et al.Survey of Supervised Joint Entity Relation Extraction Methods[J].Journal of Frontiers of Computer Science & Technology,2022,16(4):713-733.
[8]FU T J,LI P H,MA W Y.Graphrel:Modeling text as relationalgraphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computa-tional Linguistics.2019:1409-1418.
[9]YAN Z,ZHANG C,FU J,et al.A partition filter network for joint entity and relation extraction[J].arXiv:2108.12202,2021.
[10]WEI Z,SU J,WANG Y,et al.A novel cascade binary taggingframework for relational triple extraction[J].arXiv:1909.03227,2019.
[11]YUAN Y,ZHOU X,PAN S,et al.A relation-specific attention network for joint entity and relation extraction[C]//International joint Conference on Artificial Intelligence.2021.
[12]WANG Y,YU B,ZHANG Y,et al.TPLinker:Single-stage joint extraction of entities and relations through token pair linking[J].arXiv:2010.13415,2020.
[13]REN F,ZHANG L,ZHAO X,et al.A simple but effective bidirectional framework for relational triple extraction[C]//Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining.2022:824-832.
[14]REN F,ZHANG L,YIN S,et al.A Conditional Cascade Model for Relational Triple Extraction[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management.2021:3393-3397.
[15]GOYAL A,GUPTA V,KUMAR M.Recent named entity re-cognition and classification techniques:a systematic review[J].Computer Science Review,2018,29:21-43.
[16]LIU X,CHEN H,XIA W.Overview of named entity recognition[J].Journal of Contemporary Educational Research,2022,6(5):65-68.
[17]LI Q,JI H.Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2014:402-412.
[18]ZHAO K,XU H,CHENG Y,et al.Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction[J].Knowledge-Based Systems,2021,219:106888.
[19]MIWA M,BANSAL M.End-to-end relation extraction usinglstms on sequences and tree structures[J].arXiv:1601.00770,2016.
[20]ZHENG S,WANG F,BAO H,et al.Joint extraction of entities and relations based on a novel tagging scheme[J].arXiv:1706.05075,2017.
[21]WANG J,LU W.Two are better than one:Joint entity and relation extraction with table-sequence encoders[J].arXiv:2010.03851,2020.
[22]DAI D,XIAO X,LYU Y,et al.Joint extraction of entities andoverlapping relations using position-attentive sequence labeling[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:6300-6308.
[23]ZENG X,ZENG D,HE S,et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computa-tional Linguistics(Volume 1:Long Papers).2018:506-514.
[24]EBERTS M,ULGES A.Span-based joint entity and relation extraction with transformer pre-training[J].arXiv:1909.07755,2019.
[25]ZHENG H,WEN R,CHEN X,et al.PRGC:Potential relation and global correspondence based joint relational triple extraction[J].arXiv:2106.09895,2021.
[26]REN F,ZHANG L,YIN S,et al.A novel global feature-oriented relational triple extraction model based on table filling[J].ar-Xiv:2109.06705,2021.
[27]HO J,KALCHBRENNER N,WEISSENBORN D,et al.Axial attention in multidimensional transformers[J].arXiv:1912.12180,2019.
[28]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[29]SHANG Y M,HUANG H,MAO X.Onerel:Joint entity and relation extraction with one module in one step[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:11285-11293.
[30]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[31]RIEDEL S,YAO L,MCCALLUM A.Modeling relations andtheir mentions without labeled text[C]//Machine Learning and Knowledge Discovery in Databases:European Conference,ECML PKDD 2010,Barcelona,Spain,September 20-24,2010,Proceedings,Part III 21.Springer Berlin Heidelberg,2010:148-163.
[32]GARDENT C,SHIMORINA A,NARAYAN S,et al.Creatingtraining corpora for nlg micro-planning[C]//55th annual mee-ting of the Association for Computational Linguistics(ACL).2017.
[33]YU B,ZHANG Z,SHU X,et al.Joint extraction of entities and relations based on a novel decomposition strategy[J].arXiv:1909.04273,2019.
[34]XU B,WANG Q,LYU Y,et al.EmRel:Joint Representation of Entities and Embedded Relations for Multi-triple Extraction[C]//Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2022:659-665.
[1] LIN Huang, LI Bicheng. Aspect-based Sentiment Analysis Based on BERT Model and Graph Attention Network [J]. Computer Science, 2024, 51(11A): 240400018-7.
[2] ZHANG Xiaoyun, ZHAO Hui. Study on Multi-task Student Emotion Recognition Methods Based on Facial Action Units [J]. Computer Science, 2024, 51(10): 105-111.
[3] ZHOU Xueyang, FU Qiming, CHEN Jianping, LU You, WANG Yunzhe. Chemical-induced Disease Relation Extraction:Graph Reasoning Method Based on Evidence Focusing [J]. Computer Science, 2024, 51(10): 351-361.
[4] LIU Qian, BAI Zhihao, CHENG Chunling, GUI Yaocheng. Image-Text Sentiment Classification Model Based on Multi-scale Cross-modal Feature Fusion [J]. Computer Science, 2024, 51(9): 258-264.
[5] LI Guo, CHEN Chen, YANG Jing, QUN Nuo. Study on Tibetan Short Text Classification Based on DAN and FastText [J]. Computer Science, 2024, 51(6A): 230700064-5.
[6] JIANG Haoda, ZHAO Chunlei, CHEN Han, WANG Chundong. Construction Method of Domain Sentiment Lexicon Based on Improved TF-IDF and BERT [J]. Computer Science, 2024, 51(6A): 230800011-9.
[7] LI Shuai, YU Juan, WU Shaocheng. Cross-lingual Text Topic Discovery Based on Ensemble Learning [J]. Computer Science, 2024, 51(6A): 230300201-8.
[8] YIN Yanyan, WANG Keke, TIAN Jiaojiao, LI Mo, XUE Yaxin, LU Chunyu, ZHAO Yunpeng. Study on Communication Simulation of Online Hot Search Topics Based on SEIR Model [J]. Computer Science, 2024, 51(6A): 230500107-6.
[9] MENG Xiangfu, REN Quanying, YANG Dongshen, LI Keqian, YAO Keyu, ZHU Yan. Literature Classification of Individual Reports of Adverse Drug Reactions Based on BERT and CNN [J]. Computer Science, 2024, 51(6A): 230400049-6.
[10] LI Bin, WANG Haochang. Implementation and Application of Chinese Grammatical Error Diagnosis System Based on CRF [J]. Computer Science, 2024, 51(6A): 230900073-6.
[11] CHEN Haoyang, ZHANG Lei. Very Short Texts Hierarchical Classification Combining Semantic Interpretation and DeBERTa [J]. Computer Science, 2024, 51(5): 250-257.
[12] HU Binhao, ZHANG Jianpeng, CHEN Hongchang. Knowledge Graph Completion Algorithm Based on Generative Adversarial Network and Positiveand Unlabeled Learning [J]. Computer Science, 2024, 51(1): 310-315.
[13] FAN Dongxu, GUO Yi. Aspect-based Multimodal Sentiment Analysis Based on Trusted Fine-grained Alignment [J]. Computer Science, 2023, 50(12): 246-254.
[14] SUN Kaixin, LIU Bin, SU Shuguang. Medical Microscopic Image Segmentation Model Based on CNN Structure and Swin Transformer [J]. Computer Science, 2023, 50(11A): 230200119-8.
[15] LUO Yuanyuan, YANG Chunming, LI Bo, ZHANG Hui, ZHAO Xujian. Chinese Medical Named Entity Recognition Method Incorporating Machine ReadingComprehension [J]. Computer Science, 2023, 50(9): 287-294.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!