基于注意力机制的命名实体识别模型研究——以军事文本为例

计算机科学 ›› 2019, Vol. 46 ›› Issue (6A): 111-114.

基于注意力机制的命名实体识别模型研究——以军事文本为例

单义栋¹, 王衡军¹, 黄河², 闫倩³

解放军信息工程大学三院郑州450001¹;
61660部队北京100000²;
山东省军区济南250000³

出版日期:2019-06-14 发布日期:2019-07-02
作者简介:单义栋(1988-),男,硕士,主要研究方向为自然语言处理;王衡军(1973-),博士,副教授,主要研究方向为神经网络、机器学习,E-mail:2793730105@qq.com(通信作者);黄河(1982-),硕士,主要研究方向为神经网络;闫倩(1985-),主要研究方向为机器学习。

Study on Named Entity Recognition Model Based on Attention Mechanism——Taking Military Text as Example

SHAN Yi-dong¹, WANG Heng-jun¹, HUANG He², YAN Qian³

The Third Institute,PLA Information Engineering University,Zhengzhou 450001,China¹;
61660 Army,Beijing 100000,China²;
Shandong Military District,Jinan 250000,China³

Online:2019-06-14 Published:2019-07-02

摘要/Abstract

摘要： 针对双向长短时记忆网络模型提取特征不充分的特点,将字向量和词向量同时作为双向长短时记忆网络的输入,并利用注意力机制分别提取两者对当前输出有用的特征,用维特比算法约束最终输出的标签序列,构建一种新的命名实体识别模型。实验结果表明,在军事文本的命名实体识别中,该模型取得了较优的识别率。

关键词: 词向量, 注意力机制, 字向量

Abstract: Due to the insufficiency of extracting features by bi-directional long-short term memory network model,the character vector and the word vector are used as the input and the attention mechanism is used to extract the features that are useful for the current output.In this paper,a new named entity recognition model was constructed by constraining the final output tag sequence with the Viterbi algorithm.The experimental results show that the model has achieved a better recognition rate in the identification of military texts.

Key words: Attention mechanism, Character vector, Word vector

中图分类号:

TP391

单义栋, 王衡军, 黄河, 闫倩. 基于注意力机制的命名实体识别模型研究——以军事文本为例[J]. 计算机科学, 2019, 46(6A): 111-114. https://doi.org/

SHAN Yi-dong, WANG Heng-jun, HUANG He, YAN Qian. Study on Named Entity Recognition Model Based on Attention Mechanism——Taking Military Text as Example[J]. Computer Science, 2019, 46(6A): 111-114. https://doi.org/

参考文献

[1]俞鸿魁,张华平,刘群,等.基于层叠隐马尔可夫模型的中文命名实体识别[J].通信学报,2006(2):87-94.
[2]胡文博,都云程,吕学强,等.基于多层条件随机场的中文命名实体识别[J].计算机工程与应用,2009,45(1):163-165,227.
[3]PASSOS A,KUMAR V,MCCALLUM A.Lexicon Infused Phrase Embeddings for Named Entity Resolution[C]∥Proceeding of the Eighteenth Conference on Computational Language Learning,2014:78-86.
[4]CHIU J P C,NICHOLS E.Named Entity Recognition with Bidirectional LSTM-CNNs[J].ArXiv:1511.08308.
[5]COLLOBERT R,WESTON J,KARLEN M,et al.Natural Language Processing(Almost) from Scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[6]冯艳红,于红,孙庚,等.基于BLSTM的命名实体识别方法[J].计算机科学,2018,45(2):261-268.
[7]王蕾.基于神经网络的中文命名实体识别研究[D].南京:南京师范大学,2017.
[8]MNIH V,HEESS N,GRAVES A,et al.Recurrent models of visual attention[C]∥Proceedings of the 27th International Conference on Neural Information Processing System.2014:2204-2212.
[9]LUONG M T,PHAM H,MANNING C D.Effective Approa-ches to Attention-based Neural Machine Translation[J].ArXiv:1508.04025.
[10]VASWANI A,SHAZEER N,PARMAR N,et al.Attention Is All You Need[J].arXiv:1706.03762.
[11]TAN Z,WANG M,XIE J,et al.Deep Semantic Role Labeling with Self-Attention[J].ArXiv:1712.01586.
[12]谢志宁.中文命名实体识别算法研究[D].杭州:浙江大学,2017.
[13]GUL K S Q,尹继泽,潘丽敏,等.基于深度神经网络的命名实体识别方法研究[J].信息网络安全,2017(10):29-35.

相关文章 15

[1]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2]	周芳泉, 成卫青. 基于全局增强图神经网络的序列推荐 Sequence Recommendation Based on Global Enhanced Graph Neural Network 计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[3]	戴禹, 许林峰. 基于文本行匹配的跨图文本阅读方法 Cross-image Text Reading Method Based on Text Line Matching 计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032
[4]	周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[5]	熊丽琴, 曹雷, 赖俊, 陈希亮. 基于值分解的多智能体深度强化学习综述 Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization 计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[6]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7]	朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[8]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[9]	闫佳丹, 贾彩燕. 基于双图神经网络信息融合的文本分类方法 Text Classification Method Based on Information Fusion of Dual-graph Neural Network 计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[10]	汪鸣, 彭舰, 黄飞虎. 基于多时间尺度时空图网络的交通流量预测模型 Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction 计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188
[11]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[12]	金方焱, 王秀利. 融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取 Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM 计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[13]	熊罗庚, 郑尚, 邹海涛, 于化龙, 高尚. 融合双向门控循环单元和注意力机制的软件自承认技术债识别方法 Software Self-admitted Technical Debt Identification with Bidirectional Gate Recurrent Unit and Attention Mechanism 计算机科学, 2022, 49(7): 212-219. https://doi.org/10.11896/jsjkx.210500075
[14]	姜胜腾, 张亦弛, 罗鹏, 刘月玲, 曹阔, 赵海涛, 魏急波. 语义通信系统的性能度量指标分析 Analysis of Performance Metrics of Semantic Communication Systems 计算机科学, 2022, 49(7): 236-241. https://doi.org/10.11896/jsjkx.211200071
[15]	彭双, 伍江江, 陈浩, 杜春, 李军. 基于注意力神经网络的对地观测卫星星上自主任务规划方法 Satellite Onboard Observation Task Planning Based on Attention Neural Network 计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed