计算机科学 ›› 2022, Vol. 49 ›› Issue (1): 252-258.doi: 10.11896/jsjkx.210600063

所属专题: 自然语言处理 虚拟专题

• 人工智能 • 上一篇    下一篇

融入自注意力机制的深度学习情感分析方法

胡艳丽, 童谭骞, 张啸宇, 彭娟   

  1. 国防科技大学信息系统工程重点实验室 长沙410073
  • 收稿日期:2021-06-04 修回日期:2021-09-10 出版日期:2022-01-15 发布日期:2022-01-18
  • 通讯作者: 胡艳丽(huyanli@nudt.edu.cn)
  • 基金资助:
    国家自然科学基金(61302144,61902417)

Self-attention-based BGRU and CNN for Sentiment Analysis

HU Yan-li, TONG Tan-qian, ZHANG Xiao-yu, PENG Juan   

  1. Science and Technology on Information Systems Engineering Laboratory,National University of Defense Technology,Changsha 410073,China
  • Received:2021-06-04 Revised:2021-09-10 Online:2022-01-15 Published:2022-01-18
  • About author:HU Yan-li,born in 1979,associate professor,is a member of China Computer Federation.Her main research interests include text mining and knowledge engineering.
  • Supported by:
    National Natural Science Foundation of China(61302144,61902417).

摘要: 文本情感极性分析是自然语言处理的热点领域,近年来基于中文语料的情感分析方法受到了学术界的广泛关注。目前大部分基于词向量的循环神经网络与卷积神经网络模型对于文本特征的提取和保留能力不足,为此文中引入了多层自注意力机制,提出了一种结合双向门控循环单元(BGRU)和多粒度卷积神经网络的中文情感极性分析方法。该方法首先使用BGRU获取文本的序列化特征信息,然后使用自注意力机制进行初步特征筛选,将处理后的特征信息导入含有不同卷积核的卷积神经网络;再使用自注意力机制对获得的局部特征进行动态权重的调整,注重关键特征的抽取;最后经Softmax获得文本情感极性。实验结果证明,模型在两种中文语料数据集上都体现了较好的分析处理性能,其中在ChineseNLPcorpus的online_shopping_10_cats数据集上取得了92.94%的情感分类准确性,在中科院谭松波学者整理的酒店评论数据集上取得了92.75%的情感分类准确度,相比目前的主流方法,其性能均有显著的提升。

关键词: 多粒度卷积神经网络, 情感分析, 双向门控制循环单元, 自注意力机制

Abstract: Text sentiment analysis is a hot field in natural language processing.In recent years,Chinese text sentiment analysis methods have been widely investigated.Most of the recurrent neural network and convolutional neural network models based on word vectors have insufficient ability to extract and retain text features.In this paper,a Chinese sentiment polarity analysis model combining bi-directional GRU (BGRU) and multi-scale CNN is proposed.First,BGRU is utilized to extract text serialization features filtered with attention mechanism.Then the convolution neural network with distinct convolution kernels is applied to attention mechanism to adjust the dynamic weights.The text is acquired by the Softmax emotional polarity.Experiments indicates that our model outperforms the state-of-the-art methods on Chinese datasets.The accuracy of sentiment classification is 92.94% on the online_shopping_10_cats dataset of ChineseNLPcorpus,and 92.75% on the hotel review dataset compiled by Tan Songbo of Chinese Academy of Sciences,which is significantly improved compared with the current mainstream methods.

Key words: Bi-directional gated recurrent unit, Multi-scale convolution neural network, Self-attention mechanism, Sentiment analysis

中图分类号: 

  • TP391
[1]WU X H,CHEN L,WEI T T,et al.Sentiment Analysis of Chinese Short Text Based on Self-Attention and Bi-LSTM[J].Journal of Chinese Information Processing,2019,33(6):100-107.
[2]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.Aconvolutional neural network for modelling sentences[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistic Stroudsburg.PA:ACL,2014:655-665.
[3]KIM Y.Convolutional neural networks for sentence classifica-tion[C]//2014 Conference on Empirical Methods in Natural Language Processing.Doha,2014:1746-1751.
[4]LIU Y.Rsearch on english textual entail recognition based on lstm[D].Harbin:Harbin Institute of Technology,2016.
[5]WANG S,JIANG J.Learning natural language in ference with LSTM[C]//Proceedings of the Human Language Technologies:The 2016 Annual Conference of the North American Chapter of the Association for Computational Linguistics.Association for Computational Linguistics,2016:1442-1451.
[6]LI R,LIN Z,LIN H,et al.Text Emotion Analysis:A Survey[J].Journal of Computer Research and Development,2018,55(1):30-52.
[7]PANG B,LEE L,VAITHYANATHAN S.Sentiment classification using machine learn ingtechniques[C]//Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing.Pennsylvania,Stroudsburg:ACL,2002:79-86.
[8]HU M Q,LIU B.Mining and summarizing customer reviews[C]//Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York:ACM,2004:168-177.
[9]YAO T F,LOU D C.Research on semantic orientation analysis for topics in Chinese sentences[J].Journal of Chinese Information Processing,2007,21(5):73-79.
[10]DAI H L,ZHONG G J,YOU Z M,et al.Public Opinion Sentiment Big Data Analysis Ensemble Method Based on Spark[J].Computer Science,2021,48(9):118-124.
[11]BOWMAN S R,ANGELI G,POTTS C,et al.A large annotated corpus for learning natural language inference[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:632-642.
[12]ROCKTÄSCHEL T,GREFENSTETTE E,HERMANN K M,et al.Reasoning about entailment with neural attention[J].ar-Xiv:1509.06664,2015.
[13]XIAO Z,LIANG P J.Chinese sentiment analysis using bidirectional LSTM with word embedding[C]//Proceedings of the 2nd International Conference on Cloud Computing and Security.Berlin:Springer,2016:601-610.
[14]PAPPAS N,POPESCU-BELIS A.Multilingual hierarchical at-tention networks for document classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL).2017:1015-1025.
[15]LIU L F,YANG L,ZHANG S W,et al.Convolution neural networks for Chinese microblog sentiment analysis[J].Journal of Chinese Information Processing,2015,29(6):159-165.
[16]ZHANG J,DUAN L G,LI A P,et al.Fine-grained SentimentAnalysis Based on Combination of Attention and Gated Mechanism[J].Computer Science,2021,48(8):226-233.
[17]YUAN H J,ZHANG X,NIU W H,et al.Sentiment AnalysisBased on Multichannel Convolution and Bidirectional GRU with Attention Mechanism[J].Journal of Chinese Information Processing,2019,33(10):109-118.
[18]WANG L Y,LIU C H,CAI D B,et al.Chinese text sentimentanalysis based on character-level two-channel composite network[J].Application Research of Computers,2020,37(9):2674-2678.
[19]MNIH V,HEESS N,GRAVES A.Recurrent models of visual attention[C]//Advances in Neural Information Processing Systems.Montreal,Canada:NPIS,2014:2204-2212.
[20]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[21]CHENG S Y,GUO Z Y,LIU W,et al.Research on Multi-granularity Sentence Interaction Natural Language Inference Based on Attention Mechanism[J].Journal of Chinese Computer Systems,2019,40(6):1215-1220.
[22]CHO K,VAN MERRIENBOER B,GULCEHRE C,et al.Learning phrase representions using RNN encoder-decoder for statistical machine translation [C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing.Stroudsburg,PA:Association for Computational Linguistics,2014:1724-1734.
[23]LI S,ZHAO Z,HU R,et al.Analogical Reasoning on Chinese Morphological and Semantic Relations[C]//Meeting of the Association for Computational Linguistics.2018.
[24]ZAREMBA W,SUTSKEVER I,VINYALS O.Recurrent neural network regularization[J].Eprint Arxiv,2014.
[25]XIAO Z,LIANG P J.Chinese sentiment analysis using bidirectional LSTM with word embedding[C]//Proc of the 2nd International Conference on Cloud Computing and Security.Berlin:Springer,2016:601-610.
[26]GAO Y,GLOWACKA D.Deep gate recurrent neural network[J].arXiv:1604.02910,2016.
[27]WANG Y Q,HUANG M,ZHAO L,et al.Attention-basedLSTM for aspect-level sentiment classification[C]//Procee-dings of Conference on Empirical Methods in Natural Language Precessing.2016:606-615.
[28]WANG Y H,ZHANG C Y,ZHAO B L,et al.Sentiment analysis of twitter data based on CNN[J].Journal of Data Acquisition and Processing,2018,33(5):921-927.
[29]GUO B,ZHANG C X,LIU J M.Improving text classification with weighted word embeddings via a multi-channel TextCNN model[J].Neurocomputing,2019,363:366-374.
[30]CHENG Y,YE Z M,WANG M W,et al.Chinese Text Sentiment Orientation Analysis Based on Convolution Neural Network and Hierarchical Attention Network[J].Journal of Chinese Information Processing,2019,33(1):133-142.
[1] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[2] 张嘉淏, 刘峰, 齐佳音.
一种基于Bottleneck Transformer的轻量级微表情识别架构
Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer
计算机科学, 2022, 49(6A): 370-377. https://doi.org/10.11896/jsjkx.210500023
[3] 赵丹丹, 黄德根, 孟佳娜, 董宇, 张攀.
基于BERT-GRU-ATT模型的中文实体关系分类
Chinese Entity Relations Classification Based on BERT-GRU-ATT
计算机科学, 2022, 49(6): 319-325. https://doi.org/10.11896/jsjkx.210600123
[4] 丁锋, 孙晓.
基于注意力机制和BiLSTM-CRF的消极情绪意见目标抽取
Negative-emotion Opinion Target Extraction Based on Attention and BiLSTM-CRF
计算机科学, 2022, 49(2): 223-230. https://doi.org/10.11896/jsjkx.210100046
[5] 袁景凌, 丁远远, 盛德明, 李琳.
基于视觉方面注意力的图像文本情感分析模型
Image-Text Sentiment Analysis Model Based on Visual Aspect Attention
计算机科学, 2022, 49(1): 219-224. https://doi.org/10.11896/jsjkx.201000074
[6] 戴宏亮, 钟国金, 游志铭, 戴宏明.
基于Spark的舆情情感大数据分析集成方法
Public Opinion Sentiment Big Data Analysis Ensemble Method Based on Spark
计算机科学, 2021, 48(9): 118-124. https://doi.org/10.11896/jsjkx.210400280
[7] 张瑾, 段利国, 李爱萍, 郝晓燕.
基于注意力与门控机制相结合的细粒度情感分析
Fine-grained Sentiment Analysis Based on Combination of Attention and Gated Mechanism
计算机科学, 2021, 48(8): 226-233. https://doi.org/10.11896/jsjkx.200700058
[8] 徐少伟, 秦品乐, 曾建朝, 赵致楷, 高媛, 王丽芳.
基于多级特征和全局上下文的纵膈淋巴结分割算法
Mediastinal Lymph Node Segmentation Algorithm Based on Multi-level Features and Global Context
计算机科学, 2021, 48(6A): 95-100. https://doi.org/10.11896/jsjkx.200700067
[9] 史伟, 付月.
考虑语境的微博短文本挖掘:情感分析的方法
Microblog Short Text Mining Considering Context:A Method of Sentiment Analysis
计算机科学, 2021, 48(6A): 158-164. https://doi.org/10.11896/jsjkx.210200089
[10] 潘芳, 张会兵, 董俊超, 首照宇.
基于高效Transformer的中文在线课程评论方面情感分析
Aspect Sentiment Analysis of Chinese Online Course Review Based on Efficient Transformer
计算机科学, 2021, 48(6A): 264-269. https://doi.org/10.11896/jsjkx.200800116
[11] 张明阳, 王刚, 彭起, 张岩峰.
学术论文公开评审平台数据分析
Data Analysis of OpenReview
计算机科学, 2021, 48(6): 63-70. https://doi.org/10.11896/jsjkx.200500138
[12] 尹久, 池凯凯, 宦若虹.
基于ATT-DGRU的文本方面级别情感分析
Aspect-level Sentiment Analysis of Text Based on ATT-DGRU
计算机科学, 2021, 48(5): 217-224. https://doi.org/10.11896/jsjkx.200500076
[13] 王习, 张凯, 李军辉, 孔芳, 张熠天.
联合自注意力和循环网络的图像标题生成
Generation of Image Caption of Joint Self-attention and Recurrent Neural Network
计算机科学, 2021, 48(4): 157-163. https://doi.org/10.11896/jsjkx.200300146
[14] 李建兰, 潘岳, 李小聪, 刘子维, 王天宇.
基于CiteSpace的中文评论文本研究现状与趋势分析
Chinese Commentary Text Research Status and Trend Analysis Based on CiteSpace
计算机科学, 2021, 48(11A): 17-21. https://doi.org/10.11896/jsjkx.210300172
[15] 周小诗, 张梓葳, 文娟.
基于神经网络机器翻译的自然语言信息隐藏
Natural Language Steganography Based on Neural Machine Translation
计算机科学, 2021, 48(11A): 557-564. https://doi.org/10.11896/jsjkx.210100015
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!