Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (6A): 220700153-6.doi: 10.11896/jsjkx.220700153

• Artificial Intelligence • Previous Articles     Next Articles

Study on Named Entity Recognition Method Based on Knowledge Graph Enhancement

GAO Xiang1,2, TANG Jiqiang3, ZHU Junwu1, LIANG Mingxuan1,2, LI Yang1,2   

  1. 1 College of Information Engineering,Yangzhou University,Yangzhou,Jiangsu 225000,China;
    2 Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China;
    3 National Computer Network Emergency Response Technical Team/Coordination Center of China,Beijing 100029,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:GAO Xiang,born in 1996,postgraduate.His main research interests include na-tural language processing and named entity recognition. TANG Jiqiang,born in 1981,master.His main research interests include na-tural language processing and network security.
  • Supported by:
    National 242 Information Security Program(2021A008),Beijing NOVA Program(Cross-discipline,Z191100001119014),National Key Research and Development Program of China(2017YFC1700300,2017YFB1002300) and National Natural Science Foundation of China(61702234).

PDF (PC)

610

Abstract: Named entity recognition is a very basic task in natural language processing,and its purpose is to identify the corresponding entities and types from a text described in natural language.As external knowledge in the form of triples,knowledge graphs have been applied in many natural language processing tasks and achieved good results.This paper proposes an attention-aligned named entity recognition method based on knowledge graph information enhancement.Firstly,the knowledge graph information is embedded through the embedding layer and attention mechanism to obtain the representation of the knowledge graph triple information.Secondly,the sentence is obtained through BERT-BiLSTM.Then,an attention alignment module is used to assign triple weights to fuse the representation of knowledge graph information and sentence information.Finally,the prediction output of the fused representation vector is controlled by softmax,and the label of the entity is obtained.This method effectively avoids changing the semantic information of the original sentence due to the fusion of knowledge graphs,and also enables the word vectors in the sentence to have rich external knowledge.The proposedmethod achieves F1 values of 95.73% and 93.80% on the Chinese general data set MSRA and the medical domain specific data set Medicine,respectively,achieving advanced perfor-mance.

Key words: Named entity recognition, Knowledge graph enhancement, Attention mechanism, Deep learning

CLC Number: 

  • TP391
[1]RAU L F.Extracting company names from text[C]//Procee-dings The Seventh IEEE Conference on Artificial Intelligence Application.1991:29-32.
[2]XU Z,QIAN X,ZHANG Y,et al.CRF-based hybrid model for word segmentation,NER and even POS tagging[C]//Procee-dings of the Sixth SIGHAN Workshop on Chinese Language Processing.2008.
[3]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL-HLT.2019:4171-4186.
[4]TONG C,PENG H,DAI Q,et al.Improving Natural Language Understanding by Reverse Mapping Bytepair Encoding[C]//Conference on Computational Natural Language Learning.
[5]RAFFEL C,SHAZEER N,ROBERTS A,et al.Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer[J].The Journal of Machine Learning Research,2020,21(1):5485-5551.
[6]LIU Y H,OTT M,GOYAL N,et al.Roberta:A robustly optimized bert pretraining approach[J].arXiv:1907.11692,2019.
[7]FISHER J,VLACHOS A.Merge and label:A novel neural network architecture for nested NER[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics(ACL 2019).Florence,Italy,Volume 1:Long Papers,2019:5840-5850.
[8]OUYANG X,CHEN S,ZHAO H,et al.A multi-Cross Matching Network for Chinese Named Entity Linking in Short Text[J].Journal of Physics:Conference Series.IOP Publishing,2019,1325(1):012069.
[9]XIA C,ZHANG C,YANG T,et al.Multi-grained named entity recognition[C]//57th Annual Meeting of the Association for Computational Linguistics(ACL 2019).Association for Computational Linguistics(ACL),2020:1430-1440.
[10]NI J,FLORIAN R.Improving Multilingual Named Entity Re-cognition with Wikipedia Entity Type Mapping[C]//Procee-dings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1275-1284.
[11]LI W,MCCALLUM A.Rapid development of Hindi named entity recognition using conditional random fields and feature induction[J].ACM Transactions on Asian Language Information Processing(TALIP),2003,2(3):290-294.
[12]KONKOL M,KONOPÍK M.CRF-based Czech named entityrecognizer and consolidation of Czech NER research[C]//International Conference on Text,Speech and Dialogue.Berlin:Springer,2013:153-160.
[13]LI Y,SHETTY P,LIU L,et al.BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Re-cognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics.2021.
[14]COLLOBERT R,WESTON J,BOTTOU L.Natural Language Processing(almost)from scratch[J].Journal of Machine Lear-ning Research,2011,12(1):2493-2537.
[15]PENG N,DREDZE M.Improving named entity recognition for Chinese social media with word segmentation representation learning[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.2016:149-155.
[16]LIU T,YAO J G,LINC Y.Towards improving neural named entity recognition with gazetteers[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:5301-5307.
[17]LAMPLE G,BALLESTEROS M,SUBRAMANIAN S,et al.Neural Architectures for Named Entity Recognition[C]//HLT-NAACL.2016.
[18]DONG X,QIAN L,GUAN Y,et al.A multiclass classification method based on deep learning for named entity recognition in electronic medical records[C]//2016 New York Scientific Data Summit(NYSDS).IEEE,2016:1-10.
[19]SHAO Y,HARDMEIER C,NIVRE J.Multilingual named entity recognition using hybrid neural networks[C]//The Sixth Swedish Language Technology Conference(SLTC).2016.
[20]WANG Z,ZHANG J,FENG J,et al.Knowledge graph and text jointly embedding[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP).2014:1591-1601.
[21]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient Estimation of Word Representations in Vector Space[J].arXiv:1301.3781,2013.
[22]HAN X,LIU Z,SUN M.Joint representation learning of textand knowledge for knowledge graph completion[J].arXiv:1611.04125,2016.
[23]ZHANG Z,HAN X,LIU Z,et al.ERNIE:Enhanced language representation with informative entities[J].arXiv:1905.07129,2019.
[24]BOSSELUT A,RASHKIN H,SAP M,et al.COMET:Com-monsense Transformers for Automatic Knowledge Graph Construction[J].2019.
[25]JARS I,PANAGET F.Improving Spoken Language Under-standing with information retrieval and active learning methods[C]//IEEE International Conference on Acoustics.IEEE,2008.
[26]LIU W,ZHOU P,ZHAO Z,et al.K-BERT:Enabling Language Representation with Knowledge Graph[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:2901-2908.
[27]GRAVES A,MOHAMED A R,HINTON G.Speech recognition with deep recurrent neural networks[C]//International Confe-rence on Acoustics,Speech,and Signal Processing.IEEE,2013:6645-6649.
[28]WANG M,YANG L,HUA X S.MSRA-MM:Bridging research and industrial societies for multimedia information retrieval[J].Microsoft Research Asia,2009,5(6).
[29]EGAN M E,GLCKNER-PAGEL J,AMBROSE C A,et al.Calcium-pump inhibitors induce functional surface expression of Delta F508-CFTR protein in cystic fibrosis epithelial cells[J].Nature Medicine,2002,8(5):485.
[30]XU B,XU Y,LIANG J,et al.Cn-dbpedia:A never-ending chinese knowledge extraction system[C]//International Conference Industrial,Engineering and other Applications Applied Intelligent Systems.2017:428-438.
[31]WU H,CHEN P,LI W,et al.Construction of Power Communication Network Knowledge Graph with BERT-BiLSTM-CRF Model Based Entity Recognition[C]//2021 IEEE 6th International Conference on Computer and Communication Systems(ICCCS).IEEE,2021.
[32]MAO M Y,WU C,ZHONG Y X,et al.BERT named entity re-cognition model with self-attention mechanism[J].CAAI Tran-sactions on Intelligent Systems,2020,15(4):772-779.
[33]ZHANG Y,YANG J.Chinese NER using lattice LSTM[EB/OL].[2019-01-02].https://arxiv.org/pdf/1805.02023.pdf.
[34]YANG P,DONG W Y.Chinese named entity recognition me-thod based on BERT embedding[J].Computer Engineering,2020,46(4):40-45,52.
[1] ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[2] SONG Xinyang, YAN Zhiyuan, SUN Muyi, DAI Linlin, LI Qi, SUN Zhenan. Review of Talking Face Generation [J]. Computer Science, 2023, 50(8): 68-78.
[3] WANG Xu, WU Yanxia, ZHANG Xue, HONG Ruize, LI Guangsheng. Survey of Rotating Object Detection Research in Computer Vision [J]. Computer Science, 2023, 50(8): 79-92.
[4] ZHOU Ziyi, XIONG Hailing. Image Captioning Optimization Strategy Based on Deep Learning [J]. Computer Science, 2023, 50(8): 99-110.
[5] TENG Sihang, WANG Lie, LI Ya. Non-autoregressive Transformer Chinese Speech Recognition Incorporating Pronunciation- Character Representation Conversion [J]. Computer Science, 2023, 50(8): 111-117.
[6] ZHANG Xiao, DONG Hongbin. Lightweight Multi-view Stereo Integrating Coarse Cost Volume and Bilateral Grid [J]. Computer Science, 2023, 50(8): 125-132.
[7] WANG Jiahao, ZHONG Xin, LI Wenxiong, ZHAO Dexin. Human Activity Recognition with Meta-learning and Attention [J]. Computer Science, 2023, 50(8): 193-201.
[8] WANG Yu, WANG Zuchao, PAN Rui. Survey of DGA Domain Name Detection Based on Character Feature [J]. Computer Science, 2023, 50(8): 251-259.
[9] WANG Mingxia, XIONG Yun. Disease Diagnosis Prediction Algorithm Based on Contrastive Learning [J]. Computer Science, 2023, 50(7): 46-52.
[10] SHEN Zhehui, WANG Kailai, KONG Xiangjie. Exploring Station Spatio-Temporal Mobility Pattern:A Short and Long-term Traffic Prediction Framework [J]. Computer Science, 2023, 50(7): 98-106.
[11] HUO Weile, JING Tao, REN Shuang. Review of 3D Object Detection for Autonomous Driving [J]. Computer Science, 2023, 50(7): 107-118.
[12] YAN Mingqiang, YU Pengfei, LI Haiyan, LI Hongsong. Arbitrary Image Style Transfer with Consistent Semantic Style [J]. Computer Science, 2023, 50(7): 129-136.
[13] ZHOU Bo, JIANG Peifeng, DUAN Chang, LUO Yuetong. Study on Single Background Object Detection Oriented Improved-RetinaNet Model and Its Application [J]. Computer Science, 2023, 50(7): 137-142.
[14] MAO Huihui, ZHAO Xiaole, DU Shengdong, TENG Fei, LI Tianrui. Short-term Subway Passenger Flow Forecasting Based on Graphical Embedding of Temporal Knowledge [J]. Computer Science, 2023, 50(7): 213-220.
[15] LI Yuqiang, LI Linfeng, ZHU Hao, HOU Mengshu. Deep Learning-based Algorithm for Active IPv6 Address Prediction [J]. Computer Science, 2023, 50(7): 261-269.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.