Computer Science ›› 2022, Vol. 49 ›› Issue (6): 319-325.doi: 10.11896/jsjkx.210600123

• Artificial Intelligence • Previous Articles     Next Articles

Chinese Entity Relations Classification Based on BERT-GRU-ATT

ZHAO Dan-dan1,2, HUANG De-gen1, MENG Jia-na2, DONG Yu2, ZHANG Pan2   

  1. 1 School of Computer Science and Technology,Dalian University of Technology,Dalian,Liaoning 116024,China
    2 School of Computer Science and Engineering,Dalian Minzu University,Dalian,Liaoning 116600,China
  • Received:2021-06-16 Revised:2021-10-22 Online:2022-06-15 Published:2022-06-08
  • About author:ZHAO Dan-dan,born in 1975,postgra-duate,associate professor,is a member of China Computer Federation.Her main research interests include nature language processing and information extraction.
    MENG Jia-na,born in 1972,Ph.D,professor,is a member of China Computer Federation.Her main research interests include nature language processing,transfer learning and sentiment ana-lysis.
  • Supported by:
    National Key Research and Development Program of China(2020AAA008004),National Natural Science Foundation of China(U1916109,61876031) and Scientific Research Fund of Liaoning Provincial Education Department (LJYT201906).

Abstract: As the basic task of natural language processing,entity relations classification plays a critical role in tasks such as knowledge graphs,intelligent question answering,semantic web construction and so on.This paper constructs the BERT-GRU-ATT model to classify Chinese entity relations.In order to eliminate the influence of Chinese word segmentation ambiguity on entity relations classification,the pre-training model BERT(bi-directional encoder representations from transformers) is introduced as the embedding layer to better obtain the context information of Chinese characters.Then gate recurrent unit(GRU) is used to capture the long-distance dependence of entities in sentences and self-attention mechanism(ATT) is used to strengthen the weight of characters that contribute significantly to relations classification,so as to obtain better results of entity relations classification.In order to enlarge the Chinese entity relations classification corpus,we translate the SemEval2010_Task8 English entity relations evaluation corpus into Chinese.The model achieves an F1 value of 75.46% on this translation corpus,which shows the effectiveness of the proposed model.In addition,the model achieves an F1 of 80.55%on the SemEval2010-Task8 English dataset,which proves that the model has certain generalization ability to English corpus.

Key words: Chinese entity relations classification, Gate recurrent unit, Pre-training model, Self-attention mechanism

CLC Number: 

  • TP391
[1] XU J,ZHANG Z X,WU Z X.Review on Techniques of Entity Relation Extraction[J].Data Analysis and Knowledge Discovery,2008,24(8):18-23.
[2] LIU Q,LI Y,DUAN H,et al.Knowledge Graph ConstructionTechniques[J].Journal of Computer Research and Development,2016,53(3):582-600.
[3] LIU Y J.Research on Automatic Extraction of Chinese NamedEntities and Entity Relations[D].Zhengzhou:Zhengzhou University,2019.
[4] E H H,ZHANG W J,XIAO S Q,et al.Survey of Entity Relationship Extraction Based on Deep Learning[J].Journal of Software,2019,30(6):1793-1818.
[5] LI D M,ZHANG Y,LI D Y,et al.Review of Entity Relation Extraction Methods[J].Journal of Computer Research and Development,2020,57(7):1424-1448.
[6] DENG B,FAN X Z,YANG L G.Entity Relation ExtractionMethod Using Semantic Pattern[J].Computer Engineering,2007,33(10):212-214.
[7] CHE W X,LIU T,LI S.Automatic Entity Relation Extraction[J].Journal of Chinese Information Processing,2005,19(2):2-7.
[8] GUO X Y,HE T T,HU X H,et al.Chinese Named Entity Relation Extraction Based on Syntactic and Semantic Features[J].Journal of Chinese Information Processing,2014,28(6):183-189.
[9] GAO J P,ZHANG H,ZHAO X J,et al.Evolutionary Relation Extraction for Domain Knowledge in Wikipedia[J].Chinese Journal of Computers,2016,39(10):2088-2101.
[10] LIU K B,LI F,LIU L,et al.Implementation of a Kernel-Based Chinese Relation Extraction System[J].Journal of Computer Research and Development,2007,44(8):1406-1411.
[11] GUO J Y,CHEN P,YU Z T,et al.Domain Specific Chinese Semantic Relation Extraction Based on Composite Kernel[J].Journal of Chinese Information Processing,2016,30(1):24-30.
[12] YU H H,QIAN L H,ZHOU G D,et al.Chinese Semantic Rela-tion Extraction Based on Unified Syntactic and Entity Semantic Tree[J].Journal of Chinese Information Processing,2010,24(5):7-23.
[13] PETRONI F,CORRO L,GEMULLA R.Core:Context-awareopen relation extraction with factorization machines[C]//Proceedings of the 2015 Conference on Empirical Methodes in Na-tural Language Processing.Lisbon,Portugal,2015:1763-1773.
[14] QIU L K,ZANG Y.ZORE:A syntax-based system for Chinese open relation extraction[C]//Proceedings of the 2014 Confe-rence on Empirical Methods in Natural Language Processing.Doha,Qatar,2014:1870-1880.
[15] QIN B,LIU A A,LIU T.Unsupervised Chinese Open EntityRelation Extraction[J].Journal of Computer Research and Development,2015,52(5):1029-1035.
[16] ZENG D,LIU K,LAI S,et al.Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics:Technical Papers(COLING 2014).Dublin,Ireland,2014:2335-2344.
[17] XU K,FENG Y,HUANG S,et al.Semantic relation classification via convolutional neural networks with simple negative sampling[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal,2015:941-949.
[18] SANTOS C N D,XIANG B,ZHOU B.Classifying relations by ranking with convolutional neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.Beijing,China,2015:132-137.
[19] YE H,CHAO W H,LUO Z C,et al.Jointly extracting relations with class ties via effective deep ranking[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada,2017:1810-1820.
[20] BAI F,RITTER A.Structured minimally supervised learning for neural relation extraction[C]//Proceedings of the 2019 Confe-rence of the North American Chapter of the Association for Computational Linguistics.Minneapolis,Minnesota,2019:3057-3069.
[21] GAO D,PENG D L,LIU C.Entity Relation Extraction Based on CNN in Large-scale Text Data[J].Journal of Chinese Computer Systems 2018,39(5):1021-1026.
[22] SUN J D,GU X S,LI Y,et al.Chinese entity relation extraction algorithms based on COAE2016 datasets[J].Journal of Shandong University(Natural Science) 2017,52(9):7-12,18.
[23] SOCHER R,HUVAL B,MANNING C D,et al.Semantic Compositionality through Recursive Mtrix-Vector Spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Prcessing and Computational Natural Language Learning.Stroudsburg,PA,UA,2012:1201-1211.
[24] LIN Y K,SHEN S Q,LIU Z Y,et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin,Germany,2016:2124-2133.
[25] CHEN L,TIMOTHY M,DMITRIY D,et al.Self-training im-proves recurrent neural networks performance for temporal relation extraction[C]//Proceedings of the 9th International Workshop on Health Text Mining and Information Analysis.Brussels,Belgium,2018:165-176.
[26] XU Y,MOU L L,LI G,et al.Classifying relations via long short term memory networks along shortest dependency paths[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal,2015:1785-1794.
[27] NGOC T V,ADEL H,GUPTA P,et al.Combining recurrentand convolutional neural networks for relation classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics.San Diego,California,2016:534-539.
[28] SCHLICHTKRULL M S,KIPF T,BLOEM P,et al.Modeling relational data with graph convolutional networks[C]//Proceedings of the Europen semantic Web Conference on.Heraklion,Crete,Greece,2018:593-607.
[29] ZHANG Y H,QI P,MANNING C D.Graph convolution over pruned dependency trees improves relation extraction[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels,Belgium,2018:2205-2215.
[30] SONG L F,ZHANG Y,WANG Z G,et al.N-ary relation extraction using graph state LSTM[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Proces-sing.Brussels,Belgium,2018:2226-2235.
[31] GUO Z J,ZHANG Y,LU W.Attention guided graph convolutional networks for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Florence,Italy,2019:241-251.
[32] CHEN Y,ZHENG D Q,ZHAO T J.Chinese relation extraction based on Deep Belief Nets[J].Journal of Software,2012,23(10):2572-2585.
[1] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[2] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[3] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[4] LIU Shuo, WANG Geng-run, PENG Jian-hua, LI Ke. Chinese Short Text Classification Algorithm Based on Hybrid Features of Characters and Words [J]. Computer Science, 2022, 49(4): 282-287.
[5] HOU Hong-xu, SUN Shuo, WU Nier. Survey of Mongolian-Chinese Neural Machine Translation [J]. Computer Science, 2022, 49(1): 31-40.
[6] HU Yan-li, TONG Tan-qian, ZHANG Xiao-yu, PENG Juan. Self-attention-based BGRU and CNN for Sentiment Analysis [J]. Computer Science, 2022, 49(1): 252-258.
[7] WANG Xi, ZHANG Kai, LI Jun-hui, KONG Fang, ZHANG Yi-tian. Generation of Image Caption of Joint Self-attention and Recurrent Neural Network [J]. Computer Science, 2021, 48(4): 157-163.
[8] ZHOU Xiao-shi, ZHANG Zi-wei, WEN Juan. Natural Language Steganography Based on Neural Machine Translation [J]. Computer Science, 2021, 48(11A): 557-564.
[9] NI Hai-qing, LIU Dan, SHI Meng-yu. Chinese Short Text Summarization Generation Model Based on Semantic-aware [J]. Computer Science, 2020, 47(6): 74-78.
[10] ZHANG Peng-fei, LI Guan-yu, JIA Cai-yan. Truncated Gaussian Distance-based Self-attention Mechanism for Natural Language Inference [J]. Computer Science, 2020, 47(4): 178-183.
[11] ZHANG Yi-jie, LI Pei-feng, ZHU Qiao-ming. Event Temporal Relation Classification Method Based on Self-attention Mechanism [J]. Computer Science, 2019, 46(8): 244-248.
[12] FAN Zi-wei, ZHANG Min, LI Zheng-hua. BiLSTM-based Implicit Discourse Relation Classification Combining Self-attention
Mechanism and Syntactic Information
[J]. Computer Science, 2019, 46(5): 214-220.
[13] YU Mei-yu, WU Hao, GUO Xiao-yan, JIA Qi GUO He. Sequential Feature Based Sketch Recognition [J]. Computer Science, 2018, 45(11A): 198-202.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!