Computer Science ›› 2016, Vol. 43 ›› Issue (4): 224-230.doi: 10.11896/j.issn.1002-137X.2016.04.046

Previous Articles     Next Articles

Named Entity Recognition Based on Deep Belief Net

FENG Yun-tian, ZHANG Hong-jun, HAO Wen-ning and CHEN Gang   

  • Online:2018-12-01 Published:2018-12-01

Abstract: Traditional named entity recognition methods,which tag words by inputting a good deal of handmade features into statistics learning models,have achieved good results,but the manual mode of defining features makes it more difficult to build the model.To decrease the workload of the manual mode,this paper firstly got the distributed representation of word features by training the neural network language model without supervision,then discovered the deep features of words by inputting the distributed features into the deep belief net,finally conducted named entity recognition.The method uses the deep belief net to extend the neural network language model on the basis of research of predecessors,and presents a deep architecture which is available for named entity recognition.Experiments show that the me-thod applied to named entity recognition can perform better than traditional conditional random field model if both only using term feature and POS feature,and has a certain use value.

Key words: Deep belief net,Named entity recognition,Neural network language model

[1] Tjong K,Sang E F,De Meulder F.Introduction to the CoNLL-2003 shared task:Language-independent named entity recognition[C]∥Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003-Volume 4.Association for Computational Linguistics,2003:142-147
[2] McCallum A,Li W.Early results for named entity recognitionwith conditional random fields,feature induction and web-enhanced lexicons[C]∥Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003-Volume 4.Association for Computational Linguistics,2003:188-191
[3] Wang Zhi-qiang.Research on Chinese named entity recognition based on conditional random fields[D].Nanjing:Nanjing University of Science and Technology,2006(in Chinese) 王志强.基于条件随机域的中文命名实体识别研究[D].南京:南京理工大学,2006
[4] Hinton G E,Salakhutdinov R R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507
[5] Hinton G,Osindero S,Teh Y W.A fast learning algorithm for deep belief nets[J].Neural Computation,2006,18(7):1527-1554
[6] Nadeau D,Sekine S.A survey of named entity recognition and classification[J].Lingvisticae Investigationes,2007,30(1):3-26
[7] Hinton G E.Learning distributed representations of concepts[C]∥Proceedings of the Eighth Annual Conference of the Cognitive Science Cociety.1986,1:12
[8] Wang M,Manning C D.Effect of non-linear deep architecture in sequence labeling[C]∥Proceedings of the 6th International Joint Conference on Natural Language Processing (IJCNLP).2013
[9] Mansur M,Pei W,Chang B.Feature-based Neural LanguageModel and Chinese Word Segmentation[C]∥ International Joint Conference on Natural Language Processing.2013:1271-1277
[10] Bengio Y,Ducharme R,Vincent P,et al.A neural probabilistic language model[J].The Journal of Machine Learning Research,2003,3:1137-1155
[11] Mikolov T,Kombrink S,Burget L,et al.Extensions of recurrent neural network language model[C]∥2011 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2011:5528-5531
[12] Collobert R,Weston J,Bottou L,et al.Natural language processing (almost) from scratch[J].The Journal of Machine Learning Research,2011,12:2493-2537
[13] Zheng X,Chen H,Xu T.Deep Learning for Chinese Word Segmentation and POS Tagging[C]∥EMNLP.2013:647-657
[14] Pei W,Ge T,Baobao C.Maxmargin tensor neural network for chinese word segmentation[C]∥Proceedings of ACL.2014
[15] Liu J S.Monte Carlo strategies in scientific computing[M].Springer Science & Business Media,2008
[16] Hinton G.Training products of experts by minimizing contrastive divergence[J].Neural Computation,2002,14(8):1771-1800
[17] Mikolov T,Chen K,Corrado G,et al.Efficient estimation ofword representations in vector space[J].Neural Computation,2014,4:1771-1800
[18] Mikolov T,Sutskever I,Chen K,et al.Distributed representa-tions of words and phrases and their compositionality[C]∥Advances in Neural Information Processing Systems.2013:3111-3119
[19] Mikolov T,Yih W,Zweig G.Linguistic Regularities in Continuous Space Word Representations[C]∥HLT-NAACL.2013:746-751

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!