Computer Science ›› 2021, Vol. 48 ›› Issue (3): 233-238.doi: 10.11896/jsjkx.191200074

• Artificial Intelligence • Previous Articles     Next Articles

Chinese Named Entity Recognition Based on Contextualized Char Embeddings

ZHANG Dong, CHEN Wen-liang   

  1. School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215006,China
  • Received:2019-12-09 Revised:2020-05-28 Online:2021-03-15 Published:2021-03-05
  • About author:ZHANG Dong,born in 1992,postgra-duate,is a member of China Computer Federation.His main research interests include natural language processing and named entity recognition.
    CHEN Wen-liang,born in 1977,professor,doctoral supervisor,is a member of China Computer Federation.His main research interests include natural language understanding,information extraction and knowledge graph.
  • Supported by:
    National Natural Science Foundation of China(61876115).

Abstract: Named Entity Recognition (NER) is designed to identify and classify proper nouns in text.Training data for supervised learning are usually manually annotated,and it is difficult to obtain large-scale annotated data due to time-consuming and labor-intensive.In order to solve the problem of data sparseness caused by the lack of large-scale annotation corpus and the problem of polysemy of charembeddingin the Chinese NER task,this paper uses contextualized char embeddings which is pre-trained on large-scale unlabeled data to improve the performance of the Chinese NER model.Furthermore,to solve the problem of out-of-vocabulary words in named entity recognition,this paper proposes a Chinese NER system based on word language model.We use the contextualized char embeddings of generated by the language model as the input of the NER model to capture different mea-nings of Chinese characters in different contexts.In this paper,we conduct experiments on six Chinese NER datasets.The experimental results show that the proposed model can improve the performance and the average F1 improves by 4.95%.In addition,this paper further analyzes the experimental results and finds that the proposed model can achieve better results on OOV entities,and it has good performance for some special types of Chinese entity recognition.

Key words: Contextualized char vector, Language model, Named entity recognition

CLC Number: 

  • TP391.1
[1]COLLOBERT R,WESTON J,BOTTOU L,et al.Natural Language Processing (almost) from scratch [J].Journal of Machine Learning Research,2011,12(8):2493-2537.
[2]MIKOLOU T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality [C]//Proc of NIPS.Cambridge:MIT Press,2013:3111-3119.
[3]PETERS M E,NEUMANN M,IYYER M,et al.Deep contex-tualized word representations[C]//Proceedings of NAACL-HLT.New Orleans:NAACL,2018:2227-2237.
[4]SUN Z,WANG H L.Summary of research progress in named entity recognition [J].New Technology of Library and Information Service,2010,26 (6):42-47.
[5]LI L,MAO T,HUANG D,et al.Hybrid models for Chinesenamed entity recognition [C]//Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing.Sydney:ACL,2006:72-78.
[6]ZHANG S,QIN Y,WEN J,et al.Word segmentation and named entity recognition for SIGHAN Bakeoff3[C]//Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing.Sydney:ACL,2006:158-161.
[7]ZHOU J,HE L,DAI X,et al.Chinese named entity recognition with a multi-phase model[C]//Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing.Sydney:ACL,2006:213-216.
[8]HAMMERTON J.Named entity recognition with long short-term memory[C]//Proceedings of the seventh conference on Natural language learning at HLT-NAACL.Association for Computational Linguistics.Sapporo:ACL,2003:172-175.
[9]HUANG Z,XU W,YU K.Bidirectional LSTM-CRF Models for Sequence Tagging [J].arXiv:1508.01991.
[10]MA X,HOVY E.End-to-end sequence labeling via bi-directional lstm-cnns-crf [C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin:ACL,2016:7-12.
[11]LAMPLE G,BALLESTEROS M,SUBRAMANIAN S,et al.Neural Architectures for Named Entity Recognition [C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics.Berlin:ACL,2016:260-270.
[12]ZHANG Y,YANG J.Chinese NER Using Lattice LSTM [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Melbourne:ACL,2018:1554-1564.
[13]BENGIO Y,DUCHARME R,VINCENT P,et al.A neuralprobabilistic language model [J].Journal of Machine Learning Research,2003,3(2):1137-1155.
[14]HE J,WANG H.Chinese named entity recognition and wordsegmentation based on character [C]//Proceedings of the Sixth SIGHAN Workshop on Chinese Language Processing.Columbus:ACL,2008:128-132.
[15]LIU Z,ZHU C,ZHAO T.Chinese named entity recognitionwith a sequence labeling approach:based on characters,or based on words? [C]//International Conference on Intelligent Computing.Berlin:Springer,2010:634-640.
[16]LI H,HAGIWARA M,LI Q,et al.Comparison of the Impact of Word Segmentation on Name Tagging for Chinese and Japanese [C]//Proceedings of the Ninth International Conference on Language Resources and Evaluation.Reykjavik:LREC,2014:2532-2536.
[17]CHE W,LIU Y,WANG Y,et al.Towards Better UD Parsing:Deep Contextualized Word Embeddings,Ensemble,and Treebank Concatenation [C]//Proceedings of the CoNLL 2018 Shared Task:Multilingual Parsing from Raw Text to Universal Dependencies.Melbourne:ACL,2018:55-64.
[18]HOCHREITER S,SCHMIDHUBER J.Long short-term memory [J].Neural Computation,1997,9(8):1735-1780.
[19]LING W,LUÍS T,MARUJO L,et al.Finding function in form:Compositional character models for open vocabulary word representation [C]//Proceeding of the 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon:ACL,2015:1520-1530.
[20]LAFFERTY J D,MCCALLUM A,PEREIRA F C N.Condi-tional Random Fields:Probabilistic Models for Segmenting and Labeling Sequence Data[C]//Proceedings of the 18th International Conference on Machine Learning.San Francisco:Morgan Kaufmann Publishers Inc,2001:282-289.
[21]CHEN A,PENG F,SHAN R,et al.Chinese named entity recognition with conditional probabilistic models [C]//Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing.Sydney:ACL,2006:173-176.
[22]CHEN W,ZHANG Y,ISAHARA H.Chinese named entity reco-gnition with conditional random fields [C]//Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing.Sydney:ACL,2006:118-121.
[23]XU J,WEN J,SUN X,et al.A discourse-level named entity reco-gnition and relation extraction dataset for Chinese literature text [J].CoRR,2017,11(7):100-104.
[24]ZHANG S,QIN Y,WEN J,et al.Word segmentation and named entity recognition for SIGHAN Bakeoff3 [C]//Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing.Sydney:ACL,2006:158-161.
[25]YANG F,ZHANG J,LIU G,et al.Five-Stroke Based CNN-BiRNN-CRF Network for Chinese Named Entity Recognition [C]//CCF International Conference on Natural Language Processing and Chinese Computing.Hohhot:Springer,2018:184-195.
[26]ZHOU J,QU W,ZHANG F.Chinese named entity recognition via joint identification and categorization [J].Chinese journal of electronics,2013,22(2):225-230.
[1] DU Xiao-ming, YUAN Qing-bo, YANG Fan, YAO Yi, JIANG Xiang. Construction of Named Entity Recognition Corpus in Field of Military Command and Control Support [J]. Computer Science, 2022, 49(6A): 133-139.
[2] LIU Kai, ZHANG Hong-jun, CHEN Fei-qiong. Name Entity Recognition for Military Based on Domain Adaptive Embedding [J]. Computer Science, 2022, 49(1): 292-297.
[3] PAN Fang, ZHANG Hui-bing, DONG Jun-chao, SHOU Zhao-yu. Aspect Sentiment Analysis of Chinese Online Course Review Based on Efficient Transformer [J]. Computer Science, 2021, 48(6A): 264-269.
[4] DING Ling, XIANG Yang. Chinese Event Detection with Hierarchical and Multi-granularity Semantic Fusion [J]. Computer Science, 2021, 48(5): 202-208.
[5] DONG Zhe, SHAO Ruo-qi, CHEN Yu-liang, ZHAI Wei-feng. Named Entity Recognition in Food Field Based on BERT and Adversarial Training [J]. Computer Science, 2021, 48(5): 247-253.
[6] ZHOU Xiao-jin, XU Chen-ming, RUAN Tong. Multi-granularity Medical Entity Recognition for Chinese Electronic Medical Records [J]. Computer Science, 2021, 48(4): 237-242.
[7] YU Shi-yuan, GUO Shu-ming, HUANG Rui-yang, ZHANG Jian-peng, SU Ke. Overview of Nested Named Entity Recognition [J]. Computer Science, 2021, 48(11A): 1-10.
[8] ZOU Ao, HAO Wen-ning, JIN Da-wei, CHEN Gang, TIAN Yuan. Study on Text Retrieval Based on Pre-training and Deep Hash [J]. Computer Science, 2021, 48(11): 300-306.
[9] LI Zhou-jun,FAN Yu,WU Xian-jie. Survey of Natural Language Processing Pre-training Techniques [J]. Computer Science, 2020, 47(3): 162-173.
[10] TANG Guo-qiang,GAO Da-qi,RUAN Tong,YE Qi,WANG Qi. Clinical Electronic Medical Record Named Entity Recognition Incorporating Language Model and Attention Mechanism [J]. Computer Science, 2020, 47(3): 211-216.
[11] CUI Dan-dan, LIU Xiu-lei, CHEN Ruo-yu, LIU Xu-hong, LI Zhen, QI Lin. Named Entity Recognition in Field of Ancient Chinese Based on Lattice LSTM [J]. Computer Science, 2020, 47(11A): 18-23.
[12] SHI Chun-dan, QIN Lin. Chinese Named Entity Recognition Method Based on BGRU-CRF [J]. Computer Science, 2019, 46(9): 237-242.
[13] WANG Le-le,WANG Bin-qiang,LIU Jian-gang,ZHANG Jian-hui,MIAO Qi-guang. Study on Malicious Program Detection Based on Recurrent Neural Network [J]. Computer Science, 2019, 46(7): 86-90.
[14] WANG Zi-niu, JIANG Meng, GAO Jian-ling, CHEN Ya-xian. Chinese Named Entity Recognition Method Based on BERT [J]. Computer Science, 2019, 46(11A): 138-142.
[15] ZHANG Xian, BEN Ke-rong. Modified Neural Language Model and Its Application in Code Suggestion [J]. Computer Science, 2019, 46(11): 168-175.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!