Computer Science ›› 2024, Vol. 51 ›› Issue (8): 297-303.doi: 10.11896/jsjkx.230600231

• Artificial Intelligence • Previous Articles     Next Articles

Chinese Geological Entity Relation Extraction Based on RoBERTa and Weighted Graph Convolutional Networks

ZHANG Lu, DUAN Youxiang, LIU Juan, LU Yuxi   

  1. College of Computer Science and Technology,China University of Petroleum(East China),Qingdao,Shandong 266580,China
  • Received:2023-06-29 Revised:2023-11-12 Online:2024-08-15 Published:2024-08-13
  • About author:ZHANG Lu,born in 1999,postgra-duate,is a member of CCF(No.I1760G).Her main research interests include knowledge graph,relation extraction,and so on.
    DUAN Youxiang,born in 1964,Ph.D,professor,is a member of CCF(No.05290S).His main research interests include network and service computing,the application of computer technology in oil and gas field,and so on.
  • Supported by:
    Fundamental Research Funds for the Central Universities of Ministry of Education of China(20CX05017A) and Major Scientific and Technological Projects of CNPC(ZD2019-183-006).

Abstract: Knowledge is the cornerstone of big data and artificial intelligence.Knowledge graphs offer interpretability and sca-lability advantages,making them crucial in intelligent systems.Intelligent decision has urgent application demand in various fields,providing decision support and application scenarios for knowledge graphs based on data analysis and reasoning.However,constructing and applying knowledge graphs face challenges due to complex domain scenarios,multi-source data,and extensive knowledge dimensions.To address the problem of incomplete domain knowledge patterns during geological domain knowledge graph construction and the limitations of existing entity relationship extraction methods in dealing with non-Euclidean data,a graph structure-based entity relationship extraction model RoGCN-ATT is proposed.This model utilizes RoBERTa-wwm-ext-large,a Chinese pre-trained model,as the sequence encoder combined with BiLSTM to capture richer semantic information.It also employs weighted graph convolutional networks along with attention mechanisms to capture structural dependency information and enhance the extraction performance of relation triplets.Experimental results show that the F1 value reaches 78.56% on the geological dataset.Compared with other models,RoGCN-ATT effectively improves the entity-relationship extraction performance and provides strong support for the construction and application of geological knowledge maps.

Key words: Entity relation extraction, Graph convolutional networks, Dependency parsing, Attention mechanism, Geology domain

CLC Number: 

  • TP391
[1]LI C,LIU D,ZHOU D,et al.Application and Prospect of Artificial Intelligence in the Field of Geology[J].Bulletin of Mineralogy,Petrology and Geochemistry,2022,41(3):668-677.
[2]MA R X.Research on Key Technologies of Knowledge Graph Construction in Chinese Medical Field[D].Hangzhou:Zhejiang University,2023.
[3]LI X,GAO R,QIN H,et al.EINE:Relation Classification by Enhancing the Impact of Non-Entity words[C]//Proceedings of the 2022 5th International Conference on Machine Learning and Natural Language Processing.2022:68-73.
[4]GUO Q,SUN Y,LIU G,et al.Constructing Chinese historical literature knowledge graph based on BERT[C]//Web Information Systems and Applications:18th International Conference,WISA 2021,Kaifeng,China,September 24-26,2021,Proceedings 18.Springer International Publishing,2021:323-334.
[5]HUANG S B,SUN X W,LI R S.Relation Classification Me-thod Based on Cross-sentence Contextual Information for Neural Network[J].Computer Science,2022,49(S1):119-124.
[6]EBERTS M,ULGES A.An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning[C]//Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics:Main Volume.2021:3650-3660.
[7]LI Z,FU L.A Relation-Aware Span-Level Transformer Net-work for Joint Entity and Relation Extraction[C]//2022 International Joint Conference on Neural Networks(IJCNN).IEEE,2022:1-8.
[8]LI H,HOU S L,TONG Q,et al.Entity Relation Extraction Method in Weapon Field Based on DCNN and GLU[J].Computer Science,2023,50(6A):220200112-7.
[9]YU X S,LI L Y,ZHOU J L,et al.AM FRel:A method for joint extraction of entity relations in Chinese electronic medical records[J].Journal of Chongqing University of Technology(Natural Science),2024,38(2):189-197.
[10]ZHANG J L,ZHANG Y F,WANG M Q,et al.Joint extraction of Chinese entity relations based on graph convolutional neural network[J].Computer Engineering,2021,47(12):103-111.
[11]CUI Y,CHE W,LIU T,et al.Revisiting PreTrained Models for Chinese Natural Language Processing[C]//Findings of the Association for Computational Linguistics:EMNLP 2020.2020:657-668.
[12]ZHANG S,ZHENG D,HU X,et al.Bidirectional long short-term memory networks for relation classification[C]//Procee-dings of the 29th Pacific Asia Conference on Language,Informa-tion and Computation.2015:73-78.
[13]KUMAR S.A survey of deep learning methods for relation extraction[J].arXiv:1705.03645,2017.
[14]ZENG D,LIU K,LAI S,et al.Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014,the 25th International Conference on Computational Linguistics:Technical Papers.2014:2335-2344.
[15]TAKASE S,OKAZAKI N,INUI K.Modeling semantic compositionality of relational patterns[J].Engineering Applications of Artificial Intelligence,2016,50:256-264.
[16]NASAR Z,JAFFRY S W,MALIK M K.Named entit-y recognition and relation extraction:State-of-the-art[J].ACM Computing Surveys(CSUR),2021,54(1):1-39.
[17]LEI X,SONG W,FAN R,et al.Semi-supervised geological disa-sters named entity recognition using few labeled data[J].Geo-Informatica,2023,27:263-288.
[18]FAN R,WANG L,YAN J,et al.Deep learning-based named entity recognition and knowledge graph construction for geological hazards[J].ISPRS International Journal of Geo-Information,2019,9(1):1-22.
[19]LUO X,ZHOU W,WANG W,et al.Attention-based relation extraction with bidirectional gated recurrent unit and highway network in the analysis of geological data[J].IEEE Access,2017,6:5705-5715.
[20]HUANG X S,ZHU Y Q,FU L J,et al.Research on a geological entity relation extraction model for gold mine based on BERT[J].Journal of Geomechanics,2021,27(3):391-399.
[21]CHEN Z L,YUAN F,LI X H,et al.Based on BERT-BiLSTM-CRF model the named entity and relation joint extration of Chinese lithological description corpus[J].Geological Review,2022,68(2):742-750.
[22]WANG Z G,WEN H Y,LU Q,et al.Joint extraction of open entity relation in geological field[J].Computer Engineering and Design,2021,42(4):996-1005.
[23]WU X Y,DUAN Y X,CHANG L J,et al.Research on entity and relation joint extraction for geological domain[J].Computer Engineering,2023,49(3):121-127.
[24]BUNESCU R,MOONEY R.A shortest path dependency kernel for relation extraction[C]//Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing.2005:724-731.
[25]CAI R,ZHANG X,WANG H.Bidirectional recurrent convolutional neural network for relation classification[C]//Procee-dings of the 54th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2016:756-765.
[26]HENDRICKX I,KIM S N,KOZAREVA Z,et al.SemEval-2010 Task 8:Multi-Way Classification of Semantic Relations between Pairs of Nominals[C]//Proceedings of the 5th International Workshop on Semantic Evaluation.2010:33-38.
[27]ZHANG Y,QI P,MANNING C D.Graph Convolution overPruned Dependency Trees Improves Relation Extraction[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:2205-2215.
[28]YU B,MENGGE X,ZHANG Z,et al.Learning to prune dependency trees with rethinking for neural relation extraction[C]//Proceedings of the 28th International Conference on Computational Linguistics.2020:3842-3852.
[29]HONG Y,LIU Y,YANG S,et al.Improving graph convolu-tional networks based on relation-aware attention for end-to-end relation extraction[J].IEEE Access,2020,8:51315-51323.
[30]TIAN Y,CHEN G,SONG Y,et al.Dependency-driven relation extraction with attentive graph convolutional networks[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 1:Long Papers).2021:4458-4471.
[31]ZHOU H,XU Y,YAO W,et al.Global context enhanced graph convolutional networks for document-level relation extraction[C]//Proceedings of the 28th International Conference on Computational Linguistics.2020:5259-5270.
[32]DUAN J Y,YANG X,WANG H,et al.Document-level Relation Extraction of Graph Attention Convolutional Network Based on Inter-sentence Information[J].Computer Science,2023,50(S1):220800189-6.
[33]ZHAO K,XU H,CHENG Y,et al.Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction[J].Knowledge-Based Systems,2021,219:106888.
[34]ZHOU L,WANG T,QU H,et al.A weighted GCN with logical adjacency matrix for relation extraction[M]//ECAI 2020.IOS Press,2020:2314-2321.
[35]PENNINGTON J,SOCHER R,MANNING C D.Glove:Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing(EMNLP).2014:1532-1543.
[36]LI S,ZHAO Z,HU R,et al.Analogical Reasoning on Chinese Morphological and Semantic Relations[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Volume 2:Short Papers).2018:138-143.
[37]HE H,CHOI J D.The Stem Cell Hypothesis:Dilemma behind Multi-Task Learning with Transformer Encoders[C]//Procee-dings of the 2021 Conference on Empirical Methods in Natural Language Processing.2021:5555-5577.
[38]LIN Y,SHEN S,LIU Z,et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguis-tics(Volume 1:Long Papers).2016:2124-2133.
[39]MANDYA A,BOLLEGALA D,COENEN F.Graph Convolu-tion over Multiple Dependency Subgraphs for Relation Extraction[C]//COLING.International Committee on Computational Linguistics.2020:6424-6435.
[40]QI P,ZHANG Y,ZHANG Y,et al.Stanza:A Python NaturalLanguage Processing Toolkit for Many Human Languages[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics:System Demonstrations.2020:101-108.
[1] LIU Sichun, WANG Xiaoping, PEI Xilong, LUO Hangyu. Scene Segmentation Model Based on Dual Learning [J]. Computer Science, 2024, 51(8): 133-142.
[2] ZHANG Rui, WANG Ziqi, LI Yang, WANG Jiabao, CHEN Yao. Task-aware Few-shot SAR Image Classification Method Based on Multi-scale Attention Mechanism [J]. Computer Science, 2024, 51(8): 160-167.
[3] WANG Qian, HE Lang, WANG Zhanqing, HUANG Kun. Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J]. Computer Science, 2024, 51(8): 168-175.
[4] XIAO Xiao, BAI Zhengyao, LI Zekai, LIU Xuheng, DU Jiajin. Parallel Multi-scale with Attention Mechanism for Point Cloud Upsampling [J]. Computer Science, 2024, 51(8): 183-191.
[5] PU Bin, LIANG Zhengyou, SUN Yu. Monocular 3D Object Detection Based on Height-Depth Constraint and Edge Fusion [J]. Computer Science, 2024, 51(8): 192-199.
[6] ZHANG Junsan, CHENG Ming, SHEN Xiuxuan, LIU Yuxue, WANG Leiquan. Diversified Label Matrix Based Medical Image Report Generation [J]. Computer Science, 2024, 51(8): 200-208.
[7] WANG Chao, TANG Chao, WANG Wenjian, ZHANG Jing. Infrared Human Action Recognition Method Based on Multimodal Attention Network [J]. Computer Science, 2024, 51(8): 232-241.
[8] CHEN Shanshan, YAO Subin. Study on Recommendation Algorithms Based on Knowledge Graph and Neighbor PerceptionAttention Mechanism [J]. Computer Science, 2024, 51(8): 313-323.
[9] FAN Yi, HU Tao, YI Peng. Host Anomaly Detection Framework Based on Multifaceted Information Fusion of SemanticFeatures for System Calls [J]. Computer Science, 2024, 51(7): 380-388.
[10] BAI Wenchao, BAI Shuwen, HAN Xixian, ZHAO Yubo. Efficient Query Workload Prediction Algorithm Based on TCN-A [J]. Computer Science, 2024, 51(7): 71-79.
[11] ZENG Zihui, LI Chaoyang, LIAO Qing. Multivariate Time Series Anomaly Detection Algorithm in Missing Value Scenario [J]. Computer Science, 2024, 51(7): 108-115.
[12] YANG Zhenzhen, WANG Dongtao, YANG Yongpeng, HUA Renyu. Multi-embedding Fusion Based on top-N Recommendation [J]. Computer Science, 2024, 51(7): 140-145.
[13] HU Haibo, YANG Dan, NIE Tiezheng, KOU Yue. Graph Contrastive Learning Incorporating Multi-influence and Preference for Social Recommendation [J]. Computer Science, 2024, 51(7): 146-155.
[14] LI Jiaying, LIANG Yudong, LI Shaoji, ZHANG Kunpeng, ZHANG Chao. Study on Algorithm of Depth Image Super-resolution Guided by High-frequency Information ofColor Images [J]. Computer Science, 2024, 51(7): 197-205.
[15] LOU Zhengzheng, ZHANG Xin, HU Shizhe, WU Yunpeng. Foggy Weather Object Detection Method Based on YOLOX_s [J]. Computer Science, 2024, 51(7): 206-213.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!