Computer Science ›› 2019, Vol. 46 ›› Issue (6A): 111-114.

• Intelligent Computing • Previous Articles     Next Articles

Study on Named Entity Recognition Model Based on Attention Mechanism——Taking Military Text as Example

SHAN Yi-dong1, WANG Heng-jun1, HUANG He2, YAN Qian3   

  1. The Third Institute,PLA Information Engineering University,Zhengzhou 450001,China1;
    61660 Army,Beijing 100000,China2;
    Shandong Military District,Jinan 250000,China3
  • Online:2019-06-14 Published:2019-07-02

Abstract: Due to the insufficiency of extracting features by bi-directional long-short term memory network model,the character vector and the word vector are used as the input and the attention mechanism is used to extract the features that are useful for the current output.In this paper,a new named entity recognition model was constructed by constraining the final output tag sequence with the Viterbi algorithm.The experimental results show that the model has achieved a better recognition rate in the identification of military texts.

Key words: Attention mechanism, Character vector, Word vector

CLC Number: 

  • TP391
[1] 俞鸿魁,张华平,刘群,等.基于层叠隐马尔可夫模型的中文命名实体识别[J].通信学报,2006(2):87-94.
[2] 胡文博,都云程,吕学强,等.基于多层条件随机场的中文命名实体识别[J].计算机工程与应用,2009,45(1):163-165,227.
[3] PASSOS A,KUMAR V,MCCALLUM A.Lexicon Infused Phrase Embeddings for Named Entity Resolution[C]∥Proceeding of the Eighteenth Conference on Computational Language Learning,2014:78-86.
[4] CHIU J P C,NICHOLS E.Named Entity Recognition with Bidirectional LSTM-CNNs[J].ArXiv:1511.08308.
[5] COLLOBERT R,WESTON J,KARLEN M,et al.Natural Language Processing(Almost) from Scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[6] 冯艳红,于红,孙庚,等.基于BLSTM的命名实体识别方法[J].计算机科学,2018,45(2):261-268.
[7] 王蕾.基于神经网络的中文命名实体识别研究[D].南京:南京师范大学,2017.
[8] MNIH V,HEESS N,GRAVES A,et al.Recurrent models of visual attention[C]∥Proceedings of the 27th International Conference on Neural Information Processing System.2014:2204-2212.
[9] LUONG M T,PHAM H,MANNING C D.Effective Approa-ches to Attention-based Neural Machine Translation[J].ArXiv:1508.04025.
[10] VASWANI A,SHAZEER N,PARMAR N,et al.Attention Is All You Need[J].arXiv:1706.03762.
[11] TAN Z,WANG M,XIE J,et al.Deep Semantic Role Labeling with Self-Attention[J].ArXiv:1712.01586.
[12] 谢志宁.中文命名实体识别算法研究[D].杭州:浙江大学,2017.
[13] GUL K S Q,尹继泽,潘丽敏,等.基于深度神经网络的命名实体识别方法研究[J].信息网络安全,2017(10):29-35.
[1] SUN Zhong-feng, WANG Jing. RCNN-BGRU-HN Network Model for Aspect-based Sentiment Analysis [J]. Computer Science, 2019, 46(9): 223-228.
[2] SHI Chun-dan, QIN Lin. Chinese Named Entity Recognition Method Based on BGRU-CRF [J]. Computer Science, 2019, 46(9): 237-242.
[3] GUO Xu, ZHU Jing-hua. Deep Neural Network Recommendation Model Based on User Vectorization Representation and Attention Mechanism [J]. Computer Science, 2019, 46(8): 111-115.
[4] ZHANG Yi-jie, LI Pei-feng, ZHU Qiao-ming. Event Temporal Relation Classification Method Based on Self-attention Mechanism [J]. Computer Science, 2019, 46(8): 244-248.
[5] FANG Jie, LI Pei-feng, ZHU Qiao-ming. Employing Multi-attention Mechanism to Resolve Event Coreference [J]. Computer Science, 2019, 46(8): 277-281.
[6] LI Zhou-jun,WANG Chang-bao. Survey on Deep-learning-based Machine Reading Comprehension [J]. Computer Science, 2019, 46(7): 7-12.
[7] WANG Le-le,WANG Bin-qiang,LIU Jian-gang,ZHANG Jian-hui,MIAO Qi-guang. Study on Malicious Program Detection Based on Recurrent Neural Network [J]. Computer Science, 2019, 46(7): 86-90.
[8] SHEN Chen-lin, ZHANG Lu, WU Liang-qing, LI Shou-shan. Sentiment Classification Towards Question-Answering Based on Bidirectional Attention Mechanism [J]. Computer Science, 2019, 46(7): 151-156.
[9] ZHENG Cheng, HONG Tong-tong, XUE Man-yi. BLSTM_MLPCNN Model for Short Text Classification [J]. Computer Science, 2019, 46(6): 206-211.
[10] LI Jie, LING Xing-hong, FU Yu-chen, LIU Quan. Asynchronous Advantage Actor-Critic Algorithm with Visual Attention Mechanism [J]. Computer Science, 2019, 46(5): 169-174.
[11] FAN Zi-wei, ZHANG Min, LI Zheng-hua. BiLSTM-based Implicit Discourse Relation Classification Combining Self-attention
Mechanism and Syntactic Information
[J]. Computer Science, 2019, 46(5): 214-220.
[12] DENG Zhen-rong, ZHANG Bao-jun, JIANG Zhou-qin, HUANG Wen-ming. Image Description Model Fusing Word2vec and Attention Mechanism [J]. Computer Science, 2019, 46(4): 268-273.
[13] LI Hao, LIU Yong-jian, XIE Qing, TANG Ling-li. Distant Supervision Relation Extraction Model Based on Multi-level Attention Mechanism [J]. Computer Science, 2019, 46(10): 252-257.
[14] HAN Xu-li, ZENG Bi-qing, ZENG Feng, ZHANG Min, SHANG Qi. Sentiment Analysis Based on Word Embedding Auxiliary Mechanism [J]. Computer Science, 2019, 46(10): 258-264.
[15] LUO Da, SU Jin-dian, LI Peng-fei. Multi-view Attentional Approach to Single-fact Knowledge-based Question Answering [J]. Computer Science, 2019, 46(10): 215-221.
Full text



[1] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111, 142 .
[2] LI Shan and RAO Wen-bi. Video-based Detection of Human Motion Area in Mine[J]. Computer Science, 2018, 45(4): 291 -295 .
[3] QU Zhong and ZHAO Cong-mei. Anti-occlusion Adaptive-scale Object Tracking Algorithm[J]. Computer Science, 2018, 45(4): 296 -300 .
[4] LV Tao and HAO Yong-tao. Study on K-line Patterns’ Profitability Based on Similarity Match and Clustering[J]. Computer Science, 2018, 45(3): 182 -188 .
[5] ZHU Jin-bin, WU Ji-gang and SUI Xiu-feng. Edge Cloud Clustering Algorithm Based on Maximal Clique[J]. Computer Science, 2018, 45(4): 60 -65 .
[6] YU Yong,KANG Qing-yi,CHEN Chang-geng,KAN Shi-lin,LUO Yong-jun. Bisecting K-means Clustering Method Based on Cohesion and Coupling[J]. Computer Science, 2018, 45(6A): 460 -464 .
[7] FENG Yan-hong, YU Hong, SUN Geng, PENG Song. Diversity Measures Method in High-dimensional Semantic Vector Based on Asymmetric Multi-valued Feature Jaccard Coefficient[J]. Computer Science, 2018, 45(6): 57 -66 .
[8] WU Jian-xia, YANG Yong-li. Algorithm for Reducing PAPR of FBMC-OQAM System[J]. Computer Science, 2018, 45(6): 89 -95 .
[9] LIU Jing-lei, LIAO Shi-zhong. Complexity of CP-nets Learning[J]. Computer Science, 2018, 45(6): 211 -215 .
[10] HAN Li and QIAN Huan-yan. Opportunistic Routing Algorithm Combining Intra-session Coding and Inter-session Coding in Wireless Network[J]. Computer Science, 2018, 45(5): 69 -74 .