计算机科学 ›› 2021, Vol. 48 ›› Issue (11A): 154-158.doi: 10.11896/jsjkx.210100215

• 智能计算 • 上一篇    下一篇

基于双向GRU神经网络和注意力机制的中文船舶故障关系抽取方法

后同佳, 周良   

  1. 南京航空航天大学计算机科学与技术学院 南京210016
  • 出版日期:2021-11-10 发布日期:2021-11-12
  • 通讯作者: 后同佳(houtongjia@163.com)

Chinese Ship Fault Relation Extraction Method Based on Bidirectional GRU Neural Network and Attention Mechanism

HOU Tong-jia, ZHOU Liang   

  1. School of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 210016,China
  • Online:2021-11-10 Published:2021-11-12
  • About author:HOU Tong-jia,born in 1997,postgra-duate.Her main research interests include information system integration and knowledge graph.

摘要: 随着深度学习的发展,越来越多的深度学习模型被应用到了关系抽取任务中。传统的深度学习模型不能解决长距离的学习任务,且当抽取文本的噪声较大时表现更差。针对以上两个问题,提出了一种基于双向GRU(Gated Recurrent Unit)神经网络和注意力机制的深度学习模型来对中文船舶故障语料库进行关系抽取。首先,通过使用双向GRU神经网络来提取文本特征,解决了文本的长依赖问题,同时减少了模型运行的时间损耗和迭代次数;其次,通过建立句子级别的注意力机制,提高模型对有效语句的关注度,降低噪声句子给整体关系提取效果带来的负面影响;最后,在训练集上进行训练,并在真实的测试集上计算精确率、召回率、F1的值来将该模型与现有的方法对比。

关键词: 船舶故障, 关系抽取, 门控循环单元, 深度学习, 注意力机制

Abstract: With the development of deep learning,more and more deep learning models are applied to relational extraction tasks.The traditional deep learning model can not solve the long-distance learning task,and the performance of the traditional deep learning model is worse when the noise of text extraction is large.To solve the above two problems,a deep learning model based on bidirectional GRU (gated recurrent unit) neural network and attention mechanism is proposed to extract the relationship between Chinese ship faults.Firstly,by using bidirectional GRU neural network to extract text features,the problem of long dependence of text is solved,and the running time loss and iteration times of the model are also reduced.Secondly,by establishing sentence level attention mechanism,the negative impact of noisy sentences on the whole relationship extraction is reduced.Finally,the model is trained on the training set,and the accuracy,recall,and F1 values are calculated on the real test set to compare the model with existing methods.

Key words: Attention mechanism, Deep learning, Gating cycle unit, Relation extraction, Ship fault

中图分类号: 

  • TP183
[1]ZHAO Y Y,QIN B,CHE W X,et al.Research on Chinese event extraction[J].Journal of Chinese Information Processing,2008,22(1):3-8.
[2]BOLLACKER K D,EVANS C,PARITOSH P,et al.Freebase:a collaboratively created graph database for structuring human knowledge[C]//Proc of ACM SIGMOD International Confe-rence on Management of Data.New York:ACM Press,2008:1247-1250
[3]AUER S,BIZER C,KOBILAROV G,et al.DBpedia:a nucleus for a Web of open data[C]//Proc of International Semantic Web Conference.Berlin:Springer,2007:722-735
[4]YAGO:a high-quality knowledge base[EB/OL].https://www.mpiinf.mpg.De/department/databases-and-information-systems/research/yago-naga/yago/.
[5]SUN Z Y,SHUN J Z,YANG J.Chinese entity relation extraction method based on deep learning[J].Computer Engineering,2018(9):164-170.
[6]ZHANG S X.Research on automatic extraction of entity rela-tionship[J].Journal of Harbin Engineering University,2006(7):370-373.
[7]DONG J.Research on feature selection in Chinese entity relation extraction[J].Chinese Journal of information,2007(4):80-85.
[8]RINK B,HARABAGIU S.UTD:classifying semantic relations by combining lexical and semantic resources[C]//Proc of the 5th International Workshop on Semantic Evaluation.Stroudsburg,PA:Association for Computational Linguistics,2010:256-259.
[9]KAMBHATLA N.Combining lexical,syntactic,and semanticfeatures with maximum entropy models for extracting relations[C]//Proc of Interactive Poster and Demonstration Sessions.Stroudsburg,PA:Association for Computational Linguistics,2004:22.
[10]LIU K,FU H D,ZOU Y W,et al.Chinese medical weak supervision relation extraction based on convolution neural network[J].Computer Science,2017,44(10):249-253.
[11]HUANG Z W,WEI L,BIING C Z,et al.Remote supervised relation extraction based on Gru and attention mechanism[J].Computer Application Research,2019(10):76-79.
[12]ZHANG S,ZHENG D,HU,et al.Bidirectional long short-term memory networks for relation classification[C]//29th Pacific Asia Conference on Language,Information and Computation.2015:73-78.
[13]HOCHREITER S,JURGEN S.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[14]MNIH V,HEESS N.Recurrent Models of Visual Attention[J].Advances in Neural Information Processing Systems,2014,3.
[1] 周芳泉, 成卫青.
基于全局增强图神经网络的序列推荐
Sequence Recommendation Based on Global Enhanced Graph Neural Network
计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[2] 戴禹, 许林峰.
基于文本行匹配的跨图文本阅读方法
Cross-image Text Reading Method Based on Text Line Matching
计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032
[3] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[4] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[5] 熊丽琴, 曹雷, 赖俊, 陈希亮.
基于值分解的多智能体深度强化学习综述
Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization
计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[6] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[7] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[8] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[9] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[10] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[11] 汪鸣, 彭舰, 黄飞虎.
基于多时间尺度时空图网络的交通流量预测模型
Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction
计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188
[12] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[13] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[14] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[15] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!