计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220700153-6.doi: 10.11896/jsjkx.220700153

• 人工智能 • 上一篇    下一篇

基于知识增强的命名实体识别方法研究

高翔1,2, 唐积强3, 朱俊武1, 梁明轩1,2, 李阳1,2   

  1. 1 扬州大学信息工程学院 江苏 扬州 225000;
    2 中国科学院计算技术研究所 北京 100190;
    3 国家计算机网络应急技术处理协调中心 北京 100029
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 唐积强(tjq@cert.org.cn)
  • 作者简介:(stevegao66@163.com)
  • 基金资助:
    国家242信息安全计划项目(2021A008);北京市科技新星计划交叉学科合作课题(Z191100001119014);国家重点研发计划重点专项(2017YFC1700300,2017YFB1002300);国家自然科学基金(61702234)

Study on Named Entity Recognition Method Based on Knowledge Graph Enhancement

GAO Xiang1,2, TANG Jiqiang3, ZHU Junwu1, LIANG Mingxuan1,2, LI Yang1,2   

  1. 1 College of Information Engineering,Yangzhou University,Yangzhou,Jiangsu 225000,China;
    2 Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China;
    3 National Computer Network Emergency Response Technical Team/Coordination Center of China,Beijing 100029,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:GAO Xiang,born in 1996,postgraduate.His main research interests include na-tural language processing and named entity recognition. TANG Jiqiang,born in 1981,master.His main research interests include na-tural language processing and network security.
  • Supported by:
    National 242 Information Security Program(2021A008),Beijing NOVA Program(Cross-discipline,Z191100001119014),National Key Research and Development Program of China(2017YFC1700300,2017YFB1002300) and National Natural Science Foundation of China(61702234).

PDF (PC)

304

摘要： 命名实体识别作为自然语言处理中一项十分基础的任务,其目的是从一段用自然语言描述的文本中识别出相应的实体及类型。知识图谱作为以三元组形式存在的外部知识,已经在很多自然语言处理任务中得以应用并取得了良好效果。文中提出了一种基于知识图谱信息增强的注意力对齐命名实体识别方法,首先通过嵌入层和注意力机制嵌入知识图谱信息,获取知识图谱三元组信息的表示;其次通过BERT-BiLSTM获取句子的上下文表示;然后通过一种注意力对齐模块分配三元组权重融合知识图谱信息与句子信息的表示;最后通过softmax控制融合后的表示向量的预测输出,进而获取实体的标签。该方法有效避免了因知识图谱的融合而改变原句子的语义信息,同时也使得句子中的词向量具有丰富的外部知识。所提方法在中文通用数据集MSRA和医疗领域专用数据集Medicine上的F1值分别达到了95.73%和93.80%,相比基线模型提升了1.21%和1.3%。

关键词: 命名实体识别, 知识图谱增强, 注意力机制, 深度学习

Abstract: Named entity recognition is a very basic task in natural language processing,and its purpose is to identify the corresponding entities and types from a text described in natural language.As external knowledge in the form of triples,knowledge graphs have been applied in many natural language processing tasks and achieved good results.This paper proposes an attention-aligned named entity recognition method based on knowledge graph information enhancement.Firstly,the knowledge graph information is embedded through the embedding layer and attention mechanism to obtain the representation of the knowledge graph triple information.Secondly,the sentence is obtained through BERT-BiLSTM.Then,an attention alignment module is used to assign triple weights to fuse the representation of knowledge graph information and sentence information.Finally,the prediction output of the fused representation vector is controlled by softmax,and the label of the entity is obtained.This method effectively avoids changing the semantic information of the original sentence due to the fusion of knowledge graphs,and also enables the word vectors in the sentence to have rich external knowledge.The proposedmethod achieves F1 values of 95.73% and 93.80% on the Chinese general data set MSRA and the medical domain specific data set Medicine,respectively,achieving advanced perfor-mance.

Key words: Named entity recognition, Knowledge graph enhancement, Attention mechanism, Deep learning

中图分类号: 

  • TP391
[1]RAU L F.Extracting company names from text[C]//Procee-dings The Seventh IEEE Conference on Artificial Intelligence Application.1991:29-32.
[2]XU Z,QIAN X,ZHANG Y,et al.CRF-based hybrid model for word segmentation,NER and even POS tagging[C]//Procee-dings of the Sixth SIGHAN Workshop on Chinese Language Processing.2008.
[3]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL-HLT.2019:4171-4186.
[4]TONG C,PENG H,DAI Q,et al.Improving Natural Language Understanding by Reverse Mapping Bytepair Encoding[C]//Conference on Computational Natural Language Learning.
[5]RAFFEL C,SHAZEER N,ROBERTS A,et al.Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer[J].The Journal of Machine Learning Research,2020,21(1):5485-5551.
[6]LIU Y H,OTT M,GOYAL N,et al.Roberta:A robustly optimized bert pretraining approach[J].arXiv:1907.11692,2019.
[7]FISHER J,VLACHOS A.Merge and label:A novel neural network architecture for nested NER[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics(ACL 2019).Florence,Italy,Volume 1:Long Papers,2019:5840-5850.
[8]OUYANG X,CHEN S,ZHAO H,et al.A multi-Cross Matching Network for Chinese Named Entity Linking in Short Text[J].Journal of Physics:Conference Series.IOP Publishing,2019,1325(1):012069.
[9]XIA C,ZHANG C,YANG T,et al.Multi-grained named entity recognition[C]//57th Annual Meeting of the Association for Computational Linguistics(ACL 2019).Association for Computational Linguistics(ACL),2020:1430-1440.
[10]NI J,FLORIAN R.Improving Multilingual Named Entity Re-cognition with Wikipedia Entity Type Mapping[C]//Procee-dings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1275-1284.
[11]LI W,MCCALLUM A.Rapid development of Hindi named entity recognition using conditional random fields and feature induction[J].ACM Transactions on Asian Language Information Processing(TALIP),2003,2(3):290-294.
[12]KONKOL M,KONOPÍK M.CRF-based Czech named entityrecognizer and consolidation of Czech NER research[C]//International Conference on Text,Speech and Dialogue.Berlin:Springer,2013:153-160.
[13]LI Y,SHETTY P,LIU L,et al.BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Re-cognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics.2021.
[14]COLLOBERT R,WESTON J,BOTTOU L.Natural Language Processing(almost)from scratch[J].Journal of Machine Lear-ning Research,2011,12(1):2493-2537.
[15]PENG N,DREDZE M.Improving named entity recognition for Chinese social media with word segmentation representation learning[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.2016:149-155.
[16]LIU T,YAO J G,LINC Y.Towards improving neural named entity recognition with gazetteers[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:5301-5307.
[17]LAMPLE G,BALLESTEROS M,SUBRAMANIAN S,et al.Neural Architectures for Named Entity Recognition[C]//HLT-NAACL.2016.
[18]DONG X,QIAN L,GUAN Y,et al.A multiclass classification method based on deep learning for named entity recognition in electronic medical records[C]//2016 New York Scientific Data Summit(NYSDS).IEEE,2016:1-10.
[19]SHAO Y,HARDMEIER C,NIVRE J.Multilingual named entity recognition using hybrid neural networks[C]//The Sixth Swedish Language Technology Conference(SLTC).2016.
[20]WANG Z,ZHANG J,FENG J,et al.Knowledge graph and text jointly embedding[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP).2014:1591-1601.
[21]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient Estimation of Word Representations in Vector Space[J].arXiv:1301.3781,2013.
[22]HAN X,LIU Z,SUN M.Joint representation learning of textand knowledge for knowledge graph completion[J].arXiv:1611.04125,2016.
[23]ZHANG Z,HAN X,LIU Z,et al.ERNIE:Enhanced language representation with informative entities[J].arXiv:1905.07129,2019.
[24]BOSSELUT A,RASHKIN H,SAP M,et al.COMET:Com-monsense Transformers for Automatic Knowledge Graph Construction[J].2019.
[25]JARS I,PANAGET F.Improving Spoken Language Under-standing with information retrieval and active learning methods[C]//IEEE International Conference on Acoustics.IEEE,2008.
[26]LIU W,ZHOU P,ZHAO Z,et al.K-BERT:Enabling Language Representation with Knowledge Graph[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:2901-2908.
[27]GRAVES A,MOHAMED A R,HINTON G.Speech recognition with deep recurrent neural networks[C]//International Confe-rence on Acoustics,Speech,and Signal Processing.IEEE,2013:6645-6649.
[28]WANG M,YANG L,HUA X S.MSRA-MM:Bridging research and industrial societies for multimedia information retrieval[J].Microsoft Research Asia,2009,5(6).
[29]EGAN M E,GLCKNER-PAGEL J,AMBROSE C A,et al.Calcium-pump inhibitors induce functional surface expression of Delta F508-CFTR protein in cystic fibrosis epithelial cells[J].Nature Medicine,2002,8(5):485.
[30]XU B,XU Y,LIANG J,et al.Cn-dbpedia:A never-ending chinese knowledge extraction system[C]//International Conference Industrial,Engineering and other Applications Applied Intelligent Systems.2017:428-438.
[31]WU H,CHEN P,LI W,et al.Construction of Power Communication Network Knowledge Graph with BERT-BiLSTM-CRF Model Based Entity Recognition[C]//2021 IEEE 6th International Conference on Computer and Communication Systems(ICCCS).IEEE,2021.
[32]MAO M Y,WU C,ZHONG Y X,et al.BERT named entity re-cognition model with self-attention mechanism[J].CAAI Tran-sactions on Intelligent Systems,2020,15(4):772-779.
[33]ZHANG Y,YANG J.Chinese NER using lattice LSTM[EB/OL].[2019-01-02].https://arxiv.org/pdf/1805.02023.pdf.
[34]YANG P,DONG W Y.Chinese named entity recognition me-thod based on BERT embedding[J].Computer Engineering,2020,46(4):40-45,52.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发