反馈机制的实体及关系联合抽取方法

doi:10.11896/jsjkx.181102117

摘要/Abstract

摘要： 实体及关系抽取是信息抽取中的两个核心任务,是构建知识图谱的重要基石。对于实体识别和关系抽取,当前通常采取人工提取特征和规则,分独立两步实现的方法,这种方法易造成数据重复预处理和错误传播。实体识别和关系抽取两个模块存在相互关联性,实体识别是进行关系抽取的基础,实体关系抽取结果又可反馈校验实体信息。因此,文中提出无须添加人工特征和引入互反馈机制的混合神经网络模型(Mufeedback-Join Model)来完成实体及其关系的联合抽取,实现实体关系对实体识别的反馈校验机制。该模型共享Bi-LSTM特征提取层来提取文本上下文特征,依据共享层特征引入Attention机制捕获关键局部特征来完成解码,再用条件随机场CRF完成实体序列的标注任务,融合共享层特征和实体特征,并将其输入到CNN模型来实现实体关系的抽取,最后计算关系抽取结果的损失值,再联合实体识别损失值反馈修正特征提取层和实体识别模型参数。将此算法应用在实体数据集上进行实验,在同等硬件环境下,该方法可以缩短的模型训练时间,提升实体及关系抽取的准确率、召回率和F1值,联合抽取的F1值整体提升了3.91%,实体识别子模块的F1值平均提升了1.34% ,关系抽取的F1值提升了5.79%。实验数据说明,联合抽取模型可以实现两个子模块的合并,从而缩短数据处理时间和减少错误数据的传递;相互反馈的机制可以提升整体识别效果。

关键词: 反馈机制, 关系抽取, 联合抽取, 深度学习, 实体识别

Abstract: Entity and relationship extraction are two core tasks in information extraction,and are the important cornerstone of knowledge mapping.At present,entity recognition and relationship extraction usually adopt the method of extracting features and rules manually and realizing them independently in two steps.This method is easy to cause duplicate data preprocessing and error propagation.The two modules are interrelated.Entity recognition is the basis of relationship extraction.The results of entity relationship extraction can also feedback and verify entity information.Therefore,a hybrid neural network model (Mufeedback-Join Model) without adding manual features and with mutual feedback mechanism was proposed to extract entities and their relationships jointly and realize the feedback checking mechanism of entity relationship to entity recognition.The model shares Bi-LSTM feature extraction layer to extract text context features,and introduces attention mechanism to capture key parts based on shared layer features.After decoding the feature,CRF is used to complete the entity sequence labeling task.The shared layer feature and entity feature are input into CNN model to realize entity relationship extraction.Finally,the relationship extraction result loss value is calculated,and the feature extraction layer of loss value feedback correction and the parameters of entity recognition model are combined.In the same hardware environment,this method can shorten the training time of model,improve the accuracy,recall and F1 value of entity and relationship extraction,The F1 value of the joint extraction is improved by 2.91%,the entity identification sub-module F1 is increased by 1.34% on average,and the relationship extraction F1 value is increased by 5.79%.The experimental data show that the joint extraction model can merge two sub-modules to reduce data processing time and error data transmission,and the mechanism of mutual feedback can improve the overall recognition effect.

Key words: Deep learning, Entity recognition, Feedback mechanism, Joint extraction, Relation extraction

中图分类号:

TP391

马建红, 李振振, 朱怀忠, 魏字默. 反馈机制的实体及关系联合抽取方法[J]. 计算机科学, 2019, 46(12): 242-249. https://doi.org/10.11896/jsjkx.181102117

MA Jian-hong, LI Zhen-zhen, ZHU Huai-zhong, WEI Zi-mo. Entity and Relationship Joint Extraction Method of Feedback Mechanism[J]. Computer Science, 2019, 46(12): 242-249. https://doi.org/10.11896/jsjkx.181102117

参考文献

[1]SUNDHIM B M.Named Entity task definition,version 2.1 [C]//Proc.of the Sixth Message Understanding Conf.America:Morgan Kaufmann Publishers,1995:319 -332.
[2]ZHANG L,ZHAO H.Named entity recognition for Chinese microblog with convolutional neural network[C]//International Conference on Natural Computation,Fuzzy Systems and Knowledge Discovery.China:IEEE Press,2017:87-92.
[3]CHEN Y,ZHENG D Q,ZHAO T J.Chinese relation extraction based on Deep Belief Nets[J].Journal of Software,2012,23(10):2572-2585.(in Chinese)
陈宇,郑德权,赵铁军.基于Deep Belief Nets的中文名实体关系抽取[J].软件学报,2012,23(10):2572-2585.
[4]MIWA M,SASAKI Y.Modeling Joint Entity and Relation Extraction with Table Representation[C]//Conference on Empirical Methods in Natural Language Processing.Qatar:Association for Computational Linguistics ,2014:944-948.
[5]MIWA M,BANSAL M.End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures[C]//Meeting of the Association for Computational Linguistics.Germany:dblp:computer science bibliography,2016:1105-1116.
[6]ZHENG S,HAO Y,LU D,et al.Joint Entity and Relation Extraction Based on A Hybrid Neural Network[J].Neurocomputing,2017,257:1-8.
[7]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[J/OL].Computer Science,2014.https://arxiv.org/pdf/1409.0473v6.pdf.
[8]KAMBHATLA N.Combining lexical,syntactic,and semantic features with maximum entropy models for extracting relations[C]//ACL Interactive Poster Demonstration Sessions,Spain:Association for Computational Linguistics.2004:22-25.
[9]OUDAH M,SHAALAN K.NERA 2.0:Improving coverage and performance of rule-based named entity recognition for Arabic[J].Natural Language Engineering,2017,23:441-472.
[10]ZHOU G D,SU J.Named entity recognition using an HMM- based chunk tagger[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.America,2002:473-480.
[11]MCCALLUM A,LI W.Early results for named entity recognition with conditional random fields,feature induction and web-enhanced lexicons[C]//Conference on Natural Language Learning at Hlt-Naacl.Canada:Association for Computational Linguistics,2003:188-191.
[12]ZHANG H N,WU D Y,LIU Y,et al.Chinese Named Entity Recognition Based on Deep Neural Network[J].Journal of Chinese Information Processing,2017,31(4):28-35.(in Chinese)
张海楠,伍大勇,刘悦,等.基于深度神经网络的中文命名实体识别[J].中文信息学报,2017,31(4):28-35.
[13]DONG C H,ZHANG J J,ZONG C Q.Character-Based LSTM-CRF with Radical-Level Features for Chinese Named Entity Recognition[C]//International Conference on Computer Processing of Oriental Languages.Lecture Notes in Computer Science,Cham:Springer,2016:239-250.
[14]LUO L,YANG Z,YANG P,et al.An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition[J].Bioinformatics,2017,34(8):1381-1388.
[15]CHE W X,LIU T,LI S.Automatic Entity Relation Extraction[J].Journal of Chinese Information Processing,2005,19(2):1-6.(in Chinese)
车万翔,刘挺,李生.实体关系自动抽取[J].中文信息学报,2005,19(2):1-6.
[16]MA X J,GUO J Y,XIAN Y T,et al.Entity Hyponymy Acquisition and Organization Combining Word Embedding and Bootstrapping in Special Domain[J].Computer Science,2018,45(1):67-72.(in Chinese)
马晓军,郭剑毅,线岩团,等.结合词向量和Bootstrapping的领域实体上下位关系获取与组织[J].计算机科学,2018,45(1):67-72.
[17]SOCHER R,PENNINGTON J,HUANG E H,et al.Semi-supervised recursive autoencoders for predicting sentiment distributions[C]//Conference on Empirical Methods in Natural Language Processing(EMNLP 2011).UK:DBLP,2011:151-161.
[18]LAI S,XU L,LIU K,et al.Recurrent convolutional neural networks for text classification[C]//Conference of the Association for the Advancement of Artificial Intelligence (AAAI).America:DBLP,2015:2267-2273.
[19]YAN X,MOU L,LI G,et al.Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path[J].Computer Science,2015,42(1):56-61.
[20]LI J W,LUONG M T,JURAFSKY D,et al.When Are Tree Structures Necessary for Deep Learning of Representations?[J/OL].https://arxiv.org/abs/1503.00185.
[21]DAN R,YIH W T.1 Global Inference for Entity and Relation Identification via a Linear Programming Formulation[M]//Introduction to Statistical Relational Learning.MIT Press,2007:608-636.
[22]YANG B,CARDIE C.Joint Inference for Fine-grained Opinion Extraction[C]//Meeting of the Association for Computational Linguistics.Bulgaria:ACL,2013:1640-1649.
[23]SINGH S,SINGH M,CHANANA S,et al.Operation and control of a hybrid wind-diesel-battery energy system connected to micro-grid[C]//International Conference on Control,Automation,Robotics and Embedded Systems.India:IEEE Press,2013:1-6.
[24]LI Q,JI H.Incremental Joint Extraction of Entity Mentions and Relations[C]//Meeting of the Association for Computational Linguistics.America:BibSonomy,2014:402-412.
[25]DZMITRY B,KYUNGHYUN C,YOSHUA B,et al.Neural machine translation by jointly learning to align and translate [J/OL].https://arxiv.org/abs/1409.0473v2.

相关文章 15

[1]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[2]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[3]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[4]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[9]	周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[10]	金方焱, 王秀利. 融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取 Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM 计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[11]	苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[12]	胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[13]	程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[14]	王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰. 基于多源迁移学习的大坝裂缝检测 Dam Crack Detection Based on Multi-source Transfer Learning 计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[15]	楚玉春, 龚航, 王学芳, 刘培顺. 基于YOLOv4的目标检测知识蒸馏算法研究 Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4 计算机科学, 2022, 49(6A): 337-344. https://doi.org/10.11896/jsjkx.210600204

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed