计算机科学 ›› 2019, Vol. 46 ›› Issue (11A): 138-142.
王子牛1, 姜猛2, 高建瓴2, 陈娅先2
WANG Zi-niu1, JIANG Meng2, GAO Jian-ling2, CHEN Ya-xian2
摘要: 针对传统的机器学习算法对中文实体识别准确率低、高度依赖特征设计以及领域自适应能力差的问题,提出了基于BERT的神经网络方法进行命名实体识别。首先,利用大规模未标注语料对BERT进行训练,获取文本抽象特征;然后,利用BiLSTM神经网络获取序列化文本的上下文抽象特征;最后,通过CRF进行序列解码标注,提取出相应的实体。该方法结合BERT和BiLSTM-CRF模型对中文实体进行识别,以无需添加任何特征的方式在1998上半年人民日报数据集上取得了94.86%的F1值。实验表明,该方法提升了实体识别的准确率、召回率及F1值,验证了该方法的有效性。
中图分类号:
[1]SANG,ERIK F,KIM T,et al.Introduction to the CoNLL-2003 Shared Task:Language-Independent Named Entity Recognition[J].IEEE TransactionsonWireless Communications,2003,21(8):142-147. [2]BENGIO Y,SCHWENK H,SENÉCAL J S.Neural Probabilistic Language Models[J].Journal of Machine Learning Research,2001,3(6):1137-1155. [3]赵晓凡,赵丹,刘永革.利用CRF实现中文人名性别的自动识别[J].微电子学与计算机,2011,28(10):122-128. [4]林广和,张绍武,林鸿飞.基于细粒度词表示的命名实体识别研究[J].中文信息学报,2018,32(11):62-71. [5]LUO G,HUANG X J,LIN C Y,et al.Joint named entity recognition and disambiguation[C]∥Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:1030-1038. [6]ZHANG L,ZHANG Y.Big Data Analysis by Infinite Deep Neural Networks [J].Journal of Computer Research and Development,2016,53(1):68-79. [7]ZHANG S,WANG X.Automatic Recognition of Chinese Or-ganization Name Based on Conditional Random Fields[C]∥International Conference on Natural Language Processing and Knowledge Engineering,2007.IEEE,2007:229-233. [8]BORTHWICK A E.A maximum entropy approach to named entity recognition[M].New York:New York University,1999. [9]程健一,关毅,何彬.基于SVM和CRF双层分类器的英文电子病历去隐私化[J].智能计算机与应用,2016,6(6):17-19. [10]COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing (almost) fromscratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537. [11]GRAVES A.Long Short-Term Memory[M]∥Supervised Sequence Labelling with Recurrent Neural Networks.Springer Berlin Heidelberg,2012:1735-1780. [12]殷昊,徐健.基于字词融合特征的微博情绪识别方法[J].计算机科学,2018,45(S2):105-109. [13]CHO K,VAN MERRIENBOER B,GULCEHRE C,et al.Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation[J].arXiv:1406.1078v3,2014. [14]HUANG Z,XU W,YU K.BidirectionalLSTM-CRF modelsfor sequencetagging[J].arXiv:1508.01991,2015. [15]LAMPLE G,BALLESTEROS M,SUBRAMAN S,et al.Neural architectures for namedentity recognition[C]∥Proceedings of NAACL-HLT.2016:260-270. [16]CHIU J P,NICHOLS E.Namedentity recognition with bidirectional LSTM-CNNs[J].Transactions of the Association for Computational Linguistics,2015,4(10):357-370. [17]FENG Y H,HONG Y U,SUN G,et al.Named Entity Recognition Method Based on BLSTM[J].Computer Science,2018,45(2):261-268. [18]SHEN Y,YUN H,LIPTON Z C,et al.Deep Active Learning for Named Entity Recognition[J].arXiv:1707.05928v3,2018. [19]JEFFREY P,SOCHER R,MANNING C D.GloVe:Global Vectors for Word Representation[C]∥Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).2014:1532-1543. [20]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[J].arXiv:1810.04805v1,2018. [21]VASWANI A,NOAM S,et al.Attention Is All You Need[J].arXiv:1706.03762v5,2017. [22]HOCHREITER S,SCHMIDHUBER J.Long Short-term Me-mory[J].Neural Computation,1997,9(8):1735-1780. [23]GRAVES A,MOHAMED A R,HINTON G.Speech Recognition with Deep Recurrent Neural Networks[C]∥IEEE International Conference on Acoustics,Speech and Signal Processing.2013:6645-6649. [24]PETERS M E,NEUMANN M.Deep contextualized word representations[J].arXiv:1802.05365v2,2018. [25]RADFORD A,NARASIMHAN K.Improving Language Understanding by Generative Pre-Training[J/OL].https://s3-us-west2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_unders-tanding_paper.pdf. |
[1] | 于家畦, 康晓东, 白程程, 刘汉卿. 一种新的中文电子病历文本检索模型 New Text Retrieval Model of Chinese Electronic Medical Records 计算机科学, 2022, 49(6A): 32-38. https://doi.org/10.11896/jsjkx.210400198 |
[2] | 杜晓明, 袁清波, 杨帆, 姚奕, 蒋祥. 军事指控保障领域命名实体识别语料库的构建 Construction of Named Entity Recognition Corpus in Field of Military Command and Control Support 计算机科学, 2022, 49(6A): 133-139. https://doi.org/10.11896/jsjkx.210400132 |
[3] | 康雁, 吴志伟, 寇勇奇, 张兰, 谢思宇, 李浩. 融合Bert和图卷积的深度集成学习软件需求分类 Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution 计算机科学, 2022, 49(6A): 150-158. https://doi.org/10.11896/jsjkx.210500065 |
[4] | 余本功, 张子薇, 王惠灵. 一种融合多层次情感和主题信息的TS-AC-EWM在线商品排序方法 TS-AC-EWM Online Product Ranking Method Based on Multi-level Emotion and Topic Information 计算机科学, 2022, 49(6A): 165-171. https://doi.org/10.11896/jsjkx.210400238 |
[5] | 郭雨欣, 陈秀宏. 融合BERT词嵌入表示和主题信息增强的自动摘要模型 Automatic Summarization Model Combining BERT Word Embedding Representation and Topic Information Enhancement 计算机科学, 2022, 49(6): 313-318. https://doi.org/10.11896/jsjkx.210400101 |
[6] | 丁锋, 孙晓. 基于注意力机制和BiLSTM-CRF的消极情绪意见目标抽取 Negative-emotion Opinion Target Extraction Based on Attention and BiLSTM-CRF 计算机科学, 2022, 49(2): 223-230. https://doi.org/10.11896/jsjkx.210100046 |
[7] | 刘凯, 张宏军, 陈飞琼. 基于领域适应嵌入的军事命名实体识别 Name Entity Recognition for Military Based on Domain Adaptive Embedding 计算机科学, 2022, 49(1): 292-297. https://doi.org/10.11896/jsjkx.201100007 |
[8] | 程思伟, 葛唯益, 王羽, 徐建. BGCN:基于BERT和图卷积网络的触发词检测 BGCN:Trigger Detection Based on BERT and Graph Convolution Network 计算机科学, 2021, 48(7): 292-298. https://doi.org/10.11896/jsjkx.200500133 |
[9] | 董哲, 邵若琦, 陈玉梁, 翟维枫. 基于BERT和对抗训练的食品领域命名实体识别 Named Entity Recognition in Food Field Based on BERT and Adversarial Training 计算机科学, 2021, 48(5): 247-253. https://doi.org/10.11896/jsjkx.200800181 |
[10] | 陈明豪, 祝跃飞, 芦斌, 翟懿, 李玎. 基于Attention-CNN的加密流量应用类型识别 Classification of Application Type of Encrypted Traffic Based on Attention-CNN 计算机科学, 2021, 48(4): 325-332. https://doi.org/10.11896/jsjkx.200900155 |
[11] | 张栋, 陈文亮. 基于上下文相关字向量的中文命名实体识别 Chinese Named Entity Recognition Based on Contextualized Char Embeddings 计算机科学, 2021, 48(3): 233-238. https://doi.org/10.11896/jsjkx.191200074 |
[12] | 余诗媛, 郭淑明, 黄瑞阳, 张建朋, 苏珂. 嵌套命名实体识别研究进展 Overview of Nested Named Entity Recognition 计算机科学, 2021, 48(11A): 1-10. https://doi.org/10.11896/jsjkx.201100165 |
[13] | 陈德, 宋华珠, 张娟, 周泓林. 融合BERT和记忆网络的实体识别 Entity Recognition Fusing BERT and Memory Networks 计算机科学, 2021, 48(10): 91-97. https://doi.org/10.11896/jsjkx.200900015 |
[14] | 龚扣林, 周宇, 丁笠, 王永超. 基于BiLSTM模型的漏洞检测 Vulnerability Detection Using Bidirectional Long Short-term Memory Networks 计算机科学, 2020, 47(5): 295-300. https://doi.org/10.11896/jsjkx.190800046 |
[15] | 唐国强,高大启,阮彤,叶琪,王祺. 融入语言模型和注意力机制的临床电子病历命名实体识别 Clinical Electronic Medical Record Named Entity Recognition Incorporating Language Model and Attention Mechanism 计算机科学, 2020, 47(3): 211-216. https://doi.org/10.11896/jsjkx.190200259 |
|