Computer Science

Computer Science ›› 2019, Vol. 46 ›› Issue (11A): 138-142.

• Intelligent Computing • Previous Articles     Next Articles

Chinese Named Entity Recognition Method Based on BERT

WANG Zi-niu1, JIANG Meng2, GAO Jian-ling2, CHEN Ya-xian2   

  1. (Network and Information Management Center,Guizhou University,Guiyang 550025,China)1;
    (College of Big Data & Information Engineering,Guizhou University,Guiyang 550025,China)2
  • Online:2019-11-10 Published:2019-11-20

PDF (PC)

3871

Abstract: In order to solve the problems of low accuracy of traditional machine learning algorithms in Chinese entity recognition,high dependence on feature design and poor adaptability in the field,a recurrent neural network method based on bidirectional encoder representation from transformers was proposed for named entity recognition.Firstly,the BERT is trained by large-scale unlabeled corpus to obtain the abstract features of the text.Then the BiLSTM neural network is used to obtain the contextual features of the serialized text.Finally,the corresponding entities are extracted by sequence labeling with CRF.The method combines the BERT and BiLSTM-CRF models for Chinese entity recognition,and has obtained the F1 value of 94.86% on the People's Daily data set in the first half of 1998 without adding any features.Experiments show that this method improves the accuracy,recall rate and F1 value of entity recognition,indicating the effectiveness of this method.

Key words: BERT, BiLSTM, Conditional random fields, Named entity recognition, Sequence labeling

CLC Number: 

  • TP391
[1]SANG,ERIK F,KIM T,et al.Introduction to the CoNLL-2003 Shared Task:Language-Independent Named Entity Recognition[J].IEEE TransactionsonWireless Communications,2003,21(8):142-147.
[2]BENGIO Y,SCHWENK H,SENÉCAL J S.Neural Probabilistic Language Models[J].Journal of Machine Learning Research,2001,3(6):1137-1155.
[3]赵晓凡,赵丹,刘永革.利用CRF实现中文人名性别的自动识别[J].微电子学与计算机,2011,28(10):122-128.
[4]林广和,张绍武,林鸿飞.基于细粒度词表示的命名实体识别研究[J].中文信息学报,2018,32(11):62-71.
[5]LUO G,HUANG X J,LIN C Y,et al.Joint named entity recognition and disambiguation[C]∥Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:1030-1038.
[6]ZHANG L,ZHANG Y.Big Data Analysis by Infinite Deep Neural Networks [J].Journal of Computer Research and Development,2016,53(1):68-79.
[7]ZHANG S,WANG X.Automatic Recognition of Chinese Or-ganization Name Based on Conditional Random Fields[C]∥International Conference on Natural Language Processing and Knowledge Engineering,2007.IEEE,2007:229-233.
[8]BORTHWICK A E.A maximum entropy approach to named entity recognition[M].New York:New York University,1999.
[9]程健一,关毅,何彬.基于SVM和CRF双层分类器的英文电子病历去隐私化[J].智能计算机与应用,2016,6(6):17-19.
[10]COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing (almost) fromscratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[11]GRAVES A.Long Short-Term Memory[M]∥Supervised Sequence Labelling with Recurrent Neural Networks.Springer Berlin Heidelberg,2012:1735-1780.
[12]殷昊,徐健.基于字词融合特征的微博情绪识别方法[J].计算机科学,2018,45(S2):105-109.
[13]CHO K,VAN MERRIENBOER B,GULCEHRE C,et al.Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation[J].arXiv:1406.1078v3,2014.
[14]HUANG Z,XU W,YU K.BidirectionalLSTM-CRF modelsfor sequencetagging[J].arXiv:1508.01991,2015.
[15]LAMPLE G,BALLESTEROS M,SUBRAMAN S,et al.Neural architectures for namedentity recognition[C]∥Proceedings of NAACL-HLT.2016:260-270.
[16]CHIU J P,NICHOLS E.Namedentity recognition with bidirectional LSTM-CNNs[J].Transactions of the Association for Computational Linguistics,2015,4(10):357-370.
[17]FENG Y H,HONG Y U,SUN G,et al.Named Entity Recognition Method Based on BLSTM[J].Computer Science,2018,45(2):261-268.
[18]SHEN Y,YUN H,LIPTON Z C,et al.Deep Active Learning for Named Entity Recognition[J].arXiv:1707.05928v3,2018.
[19]JEFFREY P,SOCHER R,MANNING C D.GloVe:Global Vectors for Word Representation[C]∥Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).2014:1532-1543.
[20]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[J].arXiv:1810.04805v1,2018.
[21]VASWANI A,NOAM S,et al.Attention Is All You Need[J].arXiv:1706.03762v5,2017.
[22]HOCHREITER S,SCHMIDHUBER J.Long Short-term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[23]GRAVES A,MOHAMED A R,HINTON G.Speech Recognition with Deep Recurrent Neural Networks[C]∥IEEE International Conference on Acoustics,Speech and Signal Processing.2013:6645-6649.
[24]PETERS M E,NEUMANN M.Deep contextualized word representations[J].arXiv:1802.05365v2,2018.
[25]RADFORD A,NARASIMHAN K.Improving Language Understanding by Generative Pre-Training[J/OL].https://s3-us-west2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_unders-tanding_paper.pdf.
[1] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[2] YU Jia-qi, KANG Xiao-dong, BAI Cheng-cheng, LIU Han-qing. New Text Retrieval Model of Chinese Electronic Medical Records [J]. Computer Science, 2022, 49(6A): 32-38.
[3] DU Xiao-ming, YUAN Qing-bo, YANG Fan, YAO Yi, JIANG Xiang. Construction of Named Entity Recognition Corpus in Field of Military Command and Control Support [J]. Computer Science, 2022, 49(6A): 133-139.
[4] KANG Yan, WU Zhi-wei, KOU Yong-qi, ZHANG Lan, XIE Si-yu, LI Hao. Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution [J]. Computer Science, 2022, 49(6A): 150-158.
[5] YU Ben-gong, ZHANG Zi-wei, WANG Hui-ling. TS-AC-EWM Online Product Ranking Method Based on Multi-level Emotion and Topic Information [J]. Computer Science, 2022, 49(6A): 165-171.
[6] GUO Yu-xin, CHEN Xiu-hong. Automatic Summarization Model Combining BERT Word Embedding Representation and Topic Information Enhancement [J]. Computer Science, 2022, 49(6): 313-318.
[7] DING Feng, SUN Xiao. Negative-emotion Opinion Target Extraction Based on Attention and BiLSTM-CRF [J]. Computer Science, 2022, 49(2): 223-230.
[8] LIU Kai, ZHANG Hong-jun, CHEN Fei-qiong. Name Entity Recognition for Military Based on Domain Adaptive Embedding [J]. Computer Science, 2022, 49(1): 292-297.
[9] CHENG Si-wei, GE Wei-yi, WANG Yu, XU Jian. BGCN:Trigger Detection Based on BERT and Graph Convolution Network [J]. Computer Science, 2021, 48(7): 292-298.
[10] DONG Zhe, SHAO Ruo-qi, CHEN Yu-liang, ZHAI Wei-feng. Named Entity Recognition in Food Field Based on BERT and Adversarial Training [J]. Computer Science, 2021, 48(5): 247-253.
[11] ZHOU Xiao-jin, XU Chen-ming, RUAN Tong. Multi-granularity Medical Entity Recognition for Chinese Electronic Medical Records [J]. Computer Science, 2021, 48(4): 237-242.
[12] CHEN Ming-hao, ZHU Yue-fei, LU Bin, ZHAI Yi, LI Ding. Classification of Application Type of Encrypted Traffic Based on Attention-CNN [J]. Computer Science, 2021, 48(4): 325-332.
[13] SHANG Xi-xue, HAN Hai-ting, ZHU Zheng-zhou. Mechanism Design of Right to Earnings of Data Utilization Based on Evolutionary Game Model [J]. Computer Science, 2021, 48(3): 144-150.
[14] ZHANG Dong, CHEN Wen-liang. Chinese Named Entity Recognition Based on Contextualized Char Embeddings [J]. Computer Science, 2021, 48(3): 233-238.
[15] YU Shi-yuan, GUO Shu-ming, HUANG Rui-yang, ZHANG Jian-peng, SU Ke. Overview of Nested Named Entity Recognition [J]. Computer Science, 2021, 48(11A): 1-10.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.