基于深度信念网络的命名实体识别

doi:10.11896/j.issn.1002-137X.2016.04.046

摘要/Abstract

摘要： 传统的命名实体识别方法是将大量手工制定的特征输入到统计学习模型中以实现对词语的标记,能够取得较好的效果,但其手工特征制定的方式增加了模型建立的难度。为了减轻传统方法中手工特征制定的工作量,首先对神经网络语言模型进行无监督训练以得到词语特征的分布式表示,然后将分布式的特征输入到深度信念网络中以发现词语的深层特征,最后进行命名实体识别。该方法在前人研究的基础上利用深度信念网络对神经网络语言模型进行了扩展,提出了一种可用于命名实体识别的深层架构。实验表明,在仅使用词特征和词性特征的条件下,该方法用于命名实体识别的性能略优于基于条件随机场模型的方法,具有一定的使用价值。

关键词: 深度信念网络,命名实体识别,神经网络语言模型

Abstract: Traditional named entity recognition methods,which tag words by inputting a good deal of handmade features into statistics learning models,have achieved good results,but the manual mode of defining features makes it more difficult to build the model.To decrease the workload of the manual mode,this paper firstly got the distributed representation of word features by training the neural network language model without supervision,then discovered the deep features of words by inputting the distributed features into the deep belief net,finally conducted named entity recognition.The method uses the deep belief net to extend the neural network language model on the basis of research of predecessors,and presents a deep architecture which is available for named entity recognition.Experiments show that the me-thod applied to named entity recognition can perform better than traditional conditional random field model if both only using term feature and POS feature,and has a certain use value.

Key words: Deep belief net,Named entity recognition,Neural network language model

冯蕴天,张宏军,郝文宁,陈刚. 基于深度信念网络的命名实体识别[J]. 计算机科学, 2016, 43(4): 224-230. https://doi.org/10.11896/j.issn.1002-137X.2016.04.046

FENG Yun-tian, ZHANG Hong-jun, HAO Wen-ning and CHEN Gang. Named Entity Recognition Based on Deep Belief Net[J]. Computer Science, 2016, 43(4): 224-230. https://doi.org/10.11896/j.issn.1002-137X.2016.04.046

参考文献

[1] Tjong K,Sang E F,De Meulder F.Introduction to the CoNLL-2003 shared task:Language-independent named entity recognition[C]∥Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003-Volume 4.Association for Computational Linguistics,2003:142-147
[2] McCallum A,Li W.Early results for named entity recognitionwith conditional random fields,feature induction and web-enhanced lexicons[C]∥Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003-Volume 4.Association for Computational Linguistics,2003:188-191
[3] Wang Zhi-qiang.Research on Chinese named entity recognition based on conditional random fields[D].Nanjing:Nanjing University of Science and Technology,2006(in Chinese) 王志强.基于条件随机域的中文命名实体识别研究[D].南京:南京理工大学,2006
[4] Hinton G E,Salakhutdinov R R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507
[5] Hinton G,Osindero S,Teh Y W.A fast learning algorithm for deep belief nets[J].Neural Computation,2006,18(7):1527-1554
[6] Nadeau D,Sekine S.A survey of named entity recognition and classification[J].Lingvisticae Investigationes,2007,30(1):3-26
[7] Hinton G E.Learning distributed representations of concepts[C]∥Proceedings of the Eighth Annual Conference of the Cognitive Science Cociety.1986,1:12
[8] Wang M,Manning C D.Effect of non-linear deep architecture in sequence labeling[C]∥Proceedings of the 6th International Joint Conference on Natural Language Processing (IJCNLP).2013
[9] Mansur M,Pei W,Chang B.Feature-based Neural LanguageModel and Chinese Word Segmentation[C]∥ International Joint Conference on Natural Language Processing.2013:1271-1277
[10] Bengio Y,Ducharme R,Vincent P,et al.A neural probabilistic language model[J].The Journal of Machine Learning Research,2003,3:1137-1155
[11] Mikolov T,Kombrink S,Burget L,et al.Extensions of recurrent neural network language model[C]∥2011 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2011:5528-5531
[12] Collobert R,Weston J,Bottou L,et al.Natural language processing (almost) from scratch[J].The Journal of Machine Learning Research,2011,12:2493-2537
[13] Zheng X,Chen H,Xu T.Deep Learning for Chinese Word Segmentation and POS Tagging[C]∥EMNLP.2013:647-657
[14] Pei W,Ge T,Baobao C.Maxmargin tensor neural network for chinese word segmentation[C]∥Proceedings of ACL.2014
[15] Liu J S.Monte Carlo strategies in scientific computing[M].Springer Science & Business Media,2008
[16] Hinton G.Training products of experts by minimizing contrastive divergence[J].Neural Computation,2002,14(8):1771-1800
[17] Mikolov T,Chen K,Corrado G,et al.Efficient estimation ofword representations in vector space[J].Neural Computation,2014,4:1771-1800
[18] Mikolov T,Sutskever I,Chen K,et al.Distributed representa-tions of words and phrases and their compositionality[C]∥Advances in Neural Information Processing Systems.2013:3111-3119
[19] Mikolov T,Yih W,Zweig G.Linguistic Regularities in Continuous Space Word Representations[C]∥HLT-NAACL.2013:746-751

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed