Computer Science ›› 2019, Vol. 46 ›› Issue (10): 252-257.doi: 10.11896/jsjkx.180901780

• Artificial Intelligence • Previous Articles     Next Articles

Distant Supervision Relation Extraction Model Based on Multi-level Attention Mechanism

LI Hao, LIU Yong-jian, XIE Qing, TANG Ling-li   

  1. (School of Computer Science and Technology,Wuhan University of Technology,Wuhan 430070,China)
    (State Press and Publication Administration Publishing Fusion Development Key Laboratory,Wuhan 430070,China)
  • Received:2018-09-20 Revised:2018-12-19 Online:2019-10-15 Published:2019-10-21

Abstract: As one of the main tasks of information extraction,entity relation extraction aims at determining the relationship category of two entities in unstructured text.At present,the supervised method with high accuracy is limited by the need for a large number of manual tagging corpus.The distant supervision method obtains a large number of relational triples by heuristic alignment between knowledge base and text set,which is the main way to solve the large-scale relational extraction task.In order to solve the problems that the high-dimensional semantics of words in sentence context are not fully utilized and the dependency-inclusion relationship between relationships is not considered in the current research on distant supervision relation extraction,this paper proposed a multi-level attention mechanism model for distant supervision relation extraction.In this model,the high-level semantics of sentences are obtained by utilizing the bidirectional GRU(Gate Recurrent Unit) neural network to code the sentence word vectors.Then,the word-level attention is introduced to calculate the degree of correlation between two entities and the context words,thus capturing the semantic information of the entity context in sentences adequately.Next,the sentence-level attention is constructed on multiple instances to reduce the tag error annotation problem.Finally,the dependency-inclusion relationship between different relationships is automatically learned by the relation-level attention.The experimental results on FreeBase+NYT public dataset show that the introduction of word-level,sentence-level and relation-level attention mechanisms on the basis of bidirectional GRU model can improve the effect of distant supervision relation extraction.Compared with the existing mainstream methods,the multi-level attention mechanism relation extraction model obtained by integrating three levels attention mechanisms improves the accuracy and recall rate by about 4%,which achieves better relation extraction effect,thus providing a theoretical foundation for further constructing the knowledge graph and intelligent question answering applications.

Key words: Attention mechanism, Bidirectional GRU, Distant supervision, Relation extraction, Word embedding

CLC Number: 

  • TP391
[1]LIU S Y,LI B C,GUO Z Z,et al.Review of Entity Relation Extraction[J].Journal of Information Engineering University,2016,17(5):541-547.(in Chinese)
刘绍毓,李弼程,郭志刚,等.实体关系抽取研究综述[J].信息工程大学学报,2016,17(5):541-547.
[2]MINTZ M,STEVEN B,RION S,et al.Distant super vision for relation extraction without labeled data [C]//Proceedings of Joint Conference of the Meeting of the ACL .Stroudsburg:Association for Computational Linguistics,2009:1003-1011.
[3]GAN L X,WAN C X,LIU D X,et al.Chinese Named Entity Relation Extraction Based on Syntactic and Semantic Features[J].Journal of Computer Research and Development,2016,53(2):284-302.(in Chinese)
甘丽新,万常选,刘德喜,等.基于句法语义特征的中文实体关系抽取[J].计算机研究与发展,2016,53(2):284-302.
[4]CHOI S P,LEE S,JUNG H,et al.An intensive case study on kernel-based relation extraction[J].Multimedia Tools & Applications,2014,71(2):741-767.
[5]ZENG D,LIU K,LAI S,et al.Relation Classification via Convolutional Deep Neural Network[C]//Proceedings of the 25th International Conference on Computational Linguistics.2014:2335-2344.
[6]LI F L,KE J.Research Progress of Entity Relation Extraction Base on Deep Learning Framework[J].Information Science,2018,V36(3):169-176.(in Chinese)
李枫林,柯佳.基于深度学习框架的实体关系抽取研究进展[J].情报科学,2018,V36(3):169-176.
[7]HOFFMANN R,ZHANG C,LING X,et al.Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg:Association for Computational Linguistics,2011:541-550.
[8]SURDEANU M,TIBSHIRANI J,NALLAPATI R,et al.Multi-instance multi-label learning for relation extraction[C]//Proceedings of Joint Conference on Empirical Methods in Natural Language Processing and Computational Natura Language Learning.2012:455-465.
[9]BENJAMIN R,DIETRICH K.Combining Generative and Dis-criminative Model Scores for Distant Supervision[C]//Procee-dings of the 2013 Conference on Empirical Methods in Natural Language Processing.2013:24-29.
[10]ZENG D,LIU K,CHEN Y,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing.2015:1753-1762.
[11]JIANG X,WANG Q,LI P,et al.Relation Extraction with Multi-instance Multi-label Convolutional Neural Networks[C]//26th International Conference on Computational Linguistics.2016:1471-1480.
[12]LEI K,CHEN D,LI Y,et al.Cooperative Denoising for Distantly Supervised Relation Extraction [C]//Proceedings of the 27th International Conference on Computational Linguistics.2018:426-436.
[13]BAHDANAU D,CHO K,BENGIO Y,et al.Neural Machine Translation by Jointly Learning to Align and Translate[J].International conference on learning representations,2015,12(7):366-381.
[14]LIN Y,SHEN S,LIU Z,et al.Neural relation extraction with selective attention over instances[C]//Proceedings of Meeting of the Association for Computational Linguistics.2016:2124-2133.
[15]JI G,LIU K,HE S,et al.Distant Supervision for Relation Extraction with Sentence-level Attention and Entity Descriptions[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence.2017:3060-3066.
[16]HE D,ZHANG H,HAO W,et al.A Customized Attention-Based Long Short-Term Memory Network for Distant Supervised Relation Extraction[J].Neural Computation,2017,27(7):1964-1985.
[17]PENNINGTON J,SOCHER R,MANNING C.Glove:Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing.2014:1532-1543.
[18]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient Estimation of Word Representations in Vector Space[OL].https://arxiv.org/abs/1301.3781.
[19]CHUNG J,GULCEHRE C,CHO K,et al.Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling[J].arXiv:1412.3555,2014.
[20]HINTON G E,SRIVASTAVA N,KRIZHEVSKY A,et al.Improving neural networks by preventing co-adaptation of feature detectors[J].Computer Science,2012,3(4):212-223.
[21]RIEDEL S,YAO L,MCCALLUM A,et al.Modeling relations and their mentions without labeled text[C]//European Conference on Machine Learning.2010:148-163.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[3] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[4] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[5] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[8] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[9] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[10] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[11] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[12] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[13] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[14] MENG Yue-bo, MU Si-rong, LIU Guang-hui, XU Sheng-jun, HAN Jiu-qiang. Person Re-identification Method Based on GoogLeNet-GMP Based on Vector Attention Mechanism [J]. Computer Science, 2022, 49(7): 142-147.
[15] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!