Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (10): 91-97.doi: 10.11896/jsjkx.200900015

• Artificial Intelligence • Previous Articles     Next Articles

Entity Recognition Fusing BERT and Memory Networks

CHEN De, SONG Hua-zhu, ZHANG Juan, ZHOU Hong-lin   

  1. School of Computer Science and Technology,Wuhan University of Technology,Wuhan 430070,China
  • Received:2020-09-02 Revised:2020-12-06 Online:2021-10-15 Published:2021-10-18
  • About author:CHEN De,born in 1992,postgraduate.His main research interest includes na-tural language processing and so on.
    SONG Hua-zhu,born in 1970,Ph.D,associate professor,master supervisor,is a senior member of China Computer Federation.Her main research interests include artificial intelligent and data mining,semantic and knowledge abstraction.
  • Supported by:
    National Special Scientific and Technological Basic Work of the Ministry of Science and Technology(2014FY110900).

PDF (PC)

924

Abstract: Entity recognition is a sub task of information extraction.The traditional entity recognition model is used to identify entities of personnel,organization,location and name.In the real world,more types of entities must be considered,and fine-grained entity recognition is needed.At the same time,traditional entity recognition models such as BiGRU cannot make full use of the global features in a wider range.This paper presents an entity recognition model based on memory network and BERT.The pre-training language model of BERT is used for better semantic representation,and the memory network module can memorize a wider range of features.The results of entity recognition for cement clinker production corpus data show that this method can re-cognize entities and has some advantages over other traditional models.In order to further verify the model in this paper,experiments are carried out on the CLUENER2020 dataset.The results show that the optimization based on BiGRU-CRF model using BERT and memory network module can improve the effect of entity recognition.

Key words: BERT, BiGRU-CRF, Entity recognition, Memory network

CLC Number: 

  • TP391
[1]GAO B T,ZHANG Y,LIU B.BioTrHMM:Biomedical named entity recognition algorithm based on transfer learning[J].Application Research of Computers,2019,36(1):45-48.
[2]YU X,ZHANG J,QIU W S,et al.Research on medical literature risk event extraction based on sequence annotation algorithm comparison[J].Computer Applications and Software,2017,34:12.
[3]ZHOU X J,XU C M,RUAN T.Multi-granularity Medical Entity Recognition for Chinese Electronic Medical Records[J].Computer Science,2021,48(4):237-242.
[4]MCCALLUM A,LI W.Early results for named entity recognition with conditional random fields,feature induction and web-enhanced lexicons[C]//Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL.Stroudsburg:Association for Computational Linguistics,2003(4):188-191.
[5]ISOZAKI H,KAZAWA H.Efficient support vector classifiersfor named entity recognition[C]// Proceedings of the 19th International Conference on Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2002(1):1-7.
[6]LI S,LI W,COOK C,et al.Independently recurrent neural network (indrnn):Building a longer and deeper rnn[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:5457-5466.
[7]TAI K S,SOCHER R,MANNING C D.Improved SemanticRepresentations From Tree-Structured Long Short-Term Me-mory Networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1:Long Papers).2015:1556-1566.
[8]CHUNG J,GULCEHRE C,CHO K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling [EB/OL].https://arxiv.org/pdf/1412.3555.pdf,2018.
[9]PETERS M E,NEUMANN M,IYYER M,et al.Deep contextua-lized word representations [C]//Proceedings of NAACL-HLT.2018:2227-2237.
[10]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1 (Long and Short Papers).2019:4171-4186.
[11]HUANG Z,XU W,YU K.Bidirectional LSTM-CRF models for sequence tagging[EB/OL].https://arxiv.org/pdf/1508.01991.pdf,2015.
[12]STRUBELL E,VERGA P,BELANGER D,et al.Fast and Accurate Entity Recognition with Iterated Dilated Convolutions[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.2017:2670-2680.
[13]LING X,WELD D S.Fine-grained entity recognition[C]//Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence.2012:94-100.
[14]MAI K,PHAM T H,NGUYEN M T,et al.An empirical study on fine-grained named entity recognition[C]//Proceedings of the 27th International Conference on Computational Linguistics.2018:711-722.
[15]LIU X,CHEN Q,DENG C,et al.Lcqmc: A large-scale chinese question matching corpus[C]//Proceedings of the 27th International Conference on Computational Linguistics.2018:1952-1962.
[16]MILLER A,FISCH A,DODGE J,et al.Key-Value MemoryNetworks for Directly Reading Documents[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1400-1409.
[17]ZHANG H N,WU D Y,LIU Y,et al.Chinese named entity re-cognition based on deep neural network[J].Journal of Chinese Information Processing,2017,31(4):28-35.
[18]DONG Z,SHAO R Q,CHEN Y L,et al.Named Entity Recognition in Food Field Based on BERT and Adversarial Training[J].Computer Science,2021,48(5):247-253.
[19]ZHANG D,CHEN W L.Chinese Named Entity RecognitionBased on Contextualized Char Embeddings[J].Computer Science,2021,48(3):233-238.
[20]ZHANG D,WANG M T,CHEN W L.Named Entity Recognition Combining Wubi Glyphs with Contextualized Character Embeddings[J].Computer Engineering,2021,47(3):94-101.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] WANG Xin-tong, WANG Xuan, SUN Zhi-xin. Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network [J]. Computer Science, 2022, 49(8): 314-322.
[3] WANG Shan, XU Chu-yi, SHI Chun-xiang, ZHANG Ying. Study on Cloud Classification Method of Satellite Cloud Images Based on CNN-LSTM [J]. Computer Science, 2022, 49(6A): 675-679.
[4] LIU Bao-bao, YANG Jing-jing, TAO Lu, WANG He-ying. Study on Prediction of Educational Statistical Data Based on DE-LSTM Model [J]. Computer Science, 2022, 49(6A): 261-266.
[5] YU Jia-qi, KANG Xiao-dong, BAI Cheng-cheng, LIU Han-qing. New Text Retrieval Model of Chinese Electronic Medical Records [J]. Computer Science, 2022, 49(6A): 32-38.
[6] DU Xiao-ming, YUAN Qing-bo, YANG Fan, YAO Yi, JIANG Xiang. Construction of Named Entity Recognition Corpus in Field of Military Command and Control Support [J]. Computer Science, 2022, 49(6A): 133-139.
[7] KANG Yan, WU Zhi-wei, KOU Yong-qi, ZHANG Lan, XIE Si-yu, LI Hao. Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution [J]. Computer Science, 2022, 49(6A): 150-158.
[8] YU Ben-gong, ZHANG Zi-wei, WANG Hui-ling. TS-AC-EWM Online Product Ranking Method Based on Multi-level Emotion and Topic Information [J]. Computer Science, 2022, 49(6A): 165-171.
[9] GUO Yu-xin, CHEN Xiu-hong. Automatic Summarization Model Combining BERT Word Embedding Representation and Topic Information Enhancement [J]. Computer Science, 2022, 49(6): 313-318.
[10] GAO Yan-lu, XU Yuan, ZHU Qun-xiong. Predicting Electric Energy Consumption Using Sandwich Structure of Attention in Double -LSTM [J]. Computer Science, 2022, 49(3): 269-275.
[11] PAN Zhi-hao, ZENG Bi, LIAO Wen-xiong, WEI Peng-fei, WEN Song. Interactive Attention Graph Convolutional Networks for Aspect-based Sentiment Classification [J]. Computer Science, 2022, 49(3): 294-300.
[12] LIU Kai, ZHANG Hong-jun, CHEN Fei-qiong. Name Entity Recognition for Military Based on Domain Adaptive Embedding [J]. Computer Science, 2022, 49(1): 292-297.
[13] CHENG Si-wei, GE Wei-yi, WANG Yu, XU Jian. BGCN:Trigger Detection Based on BERT and Graph Convolution Network [J]. Computer Science, 2021, 48(7): 292-298.
[14] DONG Zhe, SHAO Ruo-qi, CHEN Yu-liang, ZHAI Wei-feng. Named Entity Recognition in Food Field Based on BERT and Adversarial Training [J]. Computer Science, 2021, 48(5): 247-253.
[15] ZHOU Xiao-jin, XU Chen-ming, RUAN Tong. Multi-granularity Medical Entity Recognition for Chinese Electronic Medical Records [J]. Computer Science, 2021, 48(4): 237-242.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.