Computer Science ›› 2024, Vol. 51 ›› Issue (6A): 230700071-5.doi: 10.11896/jsjkx.230700071

• Artificial Intelligenc • Previous Articles     Next Articles

Method for Entity Relation Extraction Based on Heterogeneous Graph Neural Networks and TextSemantic Enhancement

PENG Bo1, LI Yaodong1, GONG Xianfu1, LI Hao2   

  1. 1 Grid Planning and Research Center of Guangdong Power Grid Co,Guangzhou 510080,China
    2 College of Computer Science,Sichuan University,Chengdu 610065,China
  • Published:2024-06-06
  • About author:PENG Bo,born in 1991,postgraduate,engineer.His main research interest is power grid planning.
  • Supported by:
    science and Technology Proiect of China Southern PowerGrid Co. Ltd 037700KK5220042(GDKIXM20220906).

Abstract: In the era of information technology,extracting structured information from massive natural language texts has become a research hotspot.The complex knowledge information in the power system needs to be solved by constructing a knowledge graph,and entity relation extraction is the upstream information extraction task,whose completeness directly affects the effectiveness of the knowledge graph.With the continuous development of deep learning,research on using deep learning techniques to solve entity relation extraction tasks has gradually been carried out and achieved good results.However,there are still problems such as incomplete application of text semantics.This paper attempts to propose an entity relation extraction method based on heterogeneous graph neural network and text semantic enhancement to address these issues.This method uses word nodes and relationship nodes to learn semantic features and obtains initial features of the two types of nodes through BRET and pre-training tasks respectively.It uses a multi-layer graph network structure for iteration and implements the interaction between the two types of nodes by using multi-head attention mechanism for information transmission in each layer.Through experimental comparison with other models on two public datasets,this model achieves the expected effect and generally outperforms other entity relationship extraction models in various scenarios.

Key words: Deep learning, Natural language processing, Knowledge graph, Entity relation extraction, Heterogeneous graph neural networks, Text semantic enhancement

CLC Number: 

  • TP391
[1]LIU K.A survey on neural relation extraction[J].Science China Technological Sciences,2020,63(10):1971-1989.
[2]KAMBHATLA N.Combining lexical,syntactic,and semanticfeatures with maximum entropy models or information extraction[C]//Proceedings of the ACL Interactive Poster and De-monstration Sessions.2004:178-181.
[3]CULOTTA A,MCCALLUM A,BETZ J.Integrating probabilistic extraction models and data mining to iscover relations and patterns in text[C]//Proceedings of the Human Language Technology Conference of the NAACL.2006:296-303.
[4]ZHANG M,ZHANG J,SU J.Exploring syntactic features for relation extraction using a convolution tree kernel[C]//Proceedings of the Human Language Technology Conference of the NAACL.2006:288-295.
[5]CHAN Y S,ROTH D.Exploiting syntactico-semantic structures for relation extraction[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies.2011:551-560.
[6]RINK B,HARABAGIU S.Utd:Classifying semantic relationsby combining lexical and semantic resources[C]//Proceedings of the 5th International Workshop on Semantic Evaluation.2010:256-259.
[7]ZENG D,LIU K,LAI S,et al.Relation classification via convolutional deep neural network[C]//25th International Confe-rence on Computational Linguistics:Technicalpapers(COLING 2014).2014:2335-2344.
[8]SHEN Y L,ZHANG P,LI X Y,et al.Grid robuststate estimation based on multi-source data fusion[J].Guangdong Electric Power,2019,32(9):146-153.
[9]YANG S L,FU W Z,DONG J.Research on the incrementalknowledge base system in intelligent decision support system[J].Journal of Hefei University of technology,2004,27(4):339-343.
[10]XU B,ZHANG Y M.System condition-based maintenance decision-making model considering severe weather conditions[J].Guangdong Electric Power,2019,32(6):61-69.
[11]HUANG C L,WANG F,WANG J J.Intelligent online hydropower dispatching based on knowledge reasoning[J].Automation of Power Systems,2010,34(21):50-54.
[12]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL-HLT.2019.
[13]YU B,ZHANG Z,SHU X,et al.Joint extraction of entities and relations based on a novel decomposition strategy[C]//Euro-pean Conference on Artificial Intelligence.2020:2282-2289
[14]ZENG X,HE S,ZENG D,et al.Learning the extraction order of multiple relational facts in a sentence with reinforcement lear-ning[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLPIJCNLP).2019:367-377.
[15]GUPTA P,SCHÜTZE H,ANDRASSY B.Table filling multi-task recurrent neural network for joint entity and relation extraction[C]//26th International Conference on Computational Linguistics:Technical Papers(COLING 2016).2016:2537-2547.
[16]ZHANG Y,QI P,MANNING C D.Graph convolution overpruned dependency trees improves relation extraction[C]//Conference on Empirical Methods in Natural Language Processing.2018.
[17]HONG Y,LIU Y,YANG S,et al.Improving graph convolu-tional networks based on relation-aware attention for end-to-end relation extraction[J].IEEE Access,2020,8:51315-51323.
[18]ZENG X,ZENG D,HE S,et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Compu-tational Linguistics(Volume 1:Long Papers).2018:506-514.
[19]FU T J,LI P H,MA W Y.GraphRel:Modeling text as rela-tional graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Florence,Italy:Association for Computational Linguistics,2019:1409-1418.
[20]WEI Z,SU J,WANG Y,et al.A novel cascade binary tagging framework for relational triple extraction[C]//Annual Meeting of the Association for Computa-tional Linguistics.2019.
[21]ZHAO K,XU H,CHENG Y,et al.Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction[J].Knowledge-Based Systems,2021,219:106888.
[1] JIANG Zhihan, ZAN Hongying, ZHANG Li. Application of Subject Enhanced Cascade Binary Pointer Tagging Framework in Chinese Medical Entity and Relation Extraction [J]. Computer Science, 2024, 51(6A): 230800179-6.
[2] WANG Yingjie, ZHANG Chengye, BAI Fengbo, WANG Zumin. Named Entity Recognition Approach of Judicial Documents Based on Transformer [J]. Computer Science, 2024, 51(6A): 230500164-9.
[3] LI Minzhe, YIN Jibin. TCM Named Entity Recognition Model Combining BERT Model and Lexical Enhancement [J]. Computer Science, 2024, 51(6A): 230900030-6.
[4] ZHU Yuliang, LIU Juntao, RAO Ziyun, ZHANG Yi, CAO Wanhua. Knowledge Reasoning Model Combining HousE with Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230600209-8.
[5] LIANG Fang, XU Xuyao, ZHAO Kailong, ZHAO Xuanfeng, ZHANG Guijun. Remote Template Detection Algorithm and Its Application in Protein Structure Prediction [J]. Computer Science, 2024, 51(6A): 230600225-7.
[6] ZHANG Tianchi, LIU Yuxuan. Research Progress of Underwater Image Processing Based on Deep Learning [J]. Computer Science, 2024, 51(6A): 230400107-12.
[7] WANG Guogang, DONG Zhihao. Lightweight Image Semantic Segmentation Based on Attention Mechanism and Densely AdjacentPrediction [J]. Computer Science, 2024, 51(6A): 230300204-8.
[8] WANG Li, CHEN Gang, XIA Mingshan, HU Hao. DUWe:Dynamic Unknown Word Embedding Approach for Web Anomaly Detection [J]. Computer Science, 2024, 51(6A): 230300191-5.
[9] MENG Xiangfu, REN Quanying, YANG Dongshen, LI Keqian, YAO Keyu, ZHU Yan. Literature Classification of Individual Reports of Adverse Drug Reactions Based on BERT and CNN [J]. Computer Science, 2024, 51(6A): 230400049-6.
[10] LI Bin, WANG Haochang. Implementation and Application of Chinese Grammatical Error Diagnosis System Based on CRF [J]. Computer Science, 2024, 51(6A): 230900073-6.
[11] JIAO Ruodan, GAO Donghui, HUANG Yanhua, LIU Shuo, DUAN Xuanfei, WANG Rui, LIU Weidong. Study and Verification on Few-shot Evaluation Methods for AI-based Quality Inspection in Production Lines [J]. Computer Science, 2024, 51(6A): 230700086-8.
[12] HE Jing, ZHAO Rui, ZHANG Hengshuo. Visual Bibliometric Analysis of Knowledge Graph [J]. Computer Science, 2024, 51(6A): 230500123-10.
[13] HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.
[14] YANG Binxia, LUO Xudong, SUN Kaili. Recent Progress on Machine Translation Based on Pre-trained Language Models [J]. Computer Science, 2024, 51(6A): 230700112-8.
[15] TANG Xin, SUN Yufei, WANG Yujue, SHI Min, ZHU Dengming. Three Layer Knowledge Graph Architecture for Industrial Digital Twins [J]. Computer Science, 2024, 51(6A): 230400153-6.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!