Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (1): 252-258.doi: 10.11896/jsjkx.210600063

Special Issue: Natural Language Processing

• Artificial Intelligence • Previous Articles     Next Articles

Self-attention-based BGRU and CNN for Sentiment Analysis

HU Yan-li, TONG Tan-qian, ZHANG Xiao-yu, PENG Juan   

  1. Science and Technology on Information Systems Engineering Laboratory,National University of Defense Technology,Changsha 410073,China
  • Received:2021-06-04 Revised:2021-09-10 Online:2022-01-15 Published:2022-01-18
  • About author:HU Yan-li,born in 1979,associate professor,is a member of China Computer Federation.Her main research interests include text mining and knowledge engineering.
  • Supported by:
    National Natural Science Foundation of China(61302144,61902417).

PDF (PC)

1368

Abstract: Text sentiment analysis is a hot field in natural language processing.In recent years,Chinese text sentiment analysis methods have been widely investigated.Most of the recurrent neural network and convolutional neural network models based on word vectors have insufficient ability to extract and retain text features.In this paper,a Chinese sentiment polarity analysis model combining bi-directional GRU (BGRU) and multi-scale CNN is proposed.First,BGRU is utilized to extract text serialization features filtered with attention mechanism.Then the convolution neural network with distinct convolution kernels is applied to attention mechanism to adjust the dynamic weights.The text is acquired by the Softmax emotional polarity.Experiments indicates that our model outperforms the state-of-the-art methods on Chinese datasets.The accuracy of sentiment classification is 92.94% on the online_shopping_10_cats dataset of ChineseNLPcorpus,and 92.75% on the hotel review dataset compiled by Tan Songbo of Chinese Academy of Sciences,which is significantly improved compared with the current mainstream methods.

Key words: Bi-directional gated recurrent unit, Multi-scale convolution neural network, Self-attention mechanism, Sentiment analysis

CLC Number: 

  • TP391
[1]WU X H,CHEN L,WEI T T,et al.Sentiment Analysis of Chinese Short Text Based on Self-Attention and Bi-LSTM[J].Journal of Chinese Information Processing,2019,33(6):100-107.
[2]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.Aconvolutional neural network for modelling sentences[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistic Stroudsburg.PA:ACL,2014:655-665.
[3]KIM Y.Convolutional neural networks for sentence classifica-tion[C]//2014 Conference on Empirical Methods in Natural Language Processing.Doha,2014:1746-1751.
[4]LIU Y.Rsearch on english textual entail recognition based on lstm[D].Harbin:Harbin Institute of Technology,2016.
[5]WANG S,JIANG J.Learning natural language in ference with LSTM[C]//Proceedings of the Human Language Technologies:The 2016 Annual Conference of the North American Chapter of the Association for Computational Linguistics.Association for Computational Linguistics,2016:1442-1451.
[6]LI R,LIN Z,LIN H,et al.Text Emotion Analysis:A Survey[J].Journal of Computer Research and Development,2018,55(1):30-52.
[7]PANG B,LEE L,VAITHYANATHAN S.Sentiment classification using machine learn ingtechniques[C]//Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing.Pennsylvania,Stroudsburg:ACL,2002:79-86.
[8]HU M Q,LIU B.Mining and summarizing customer reviews[C]//Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York:ACM,2004:168-177.
[9]YAO T F,LOU D C.Research on semantic orientation analysis for topics in Chinese sentences[J].Journal of Chinese Information Processing,2007,21(5):73-79.
[10]DAI H L,ZHONG G J,YOU Z M,et al.Public Opinion Sentiment Big Data Analysis Ensemble Method Based on Spark[J].Computer Science,2021,48(9):118-124.
[11]BOWMAN S R,ANGELI G,POTTS C,et al.A large annotated corpus for learning natural language inference[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:632-642.
[12]ROCKTÄSCHEL T,GREFENSTETTE E,HERMANN K M,et al.Reasoning about entailment with neural attention[J].ar-Xiv:1509.06664,2015.
[13]XIAO Z,LIANG P J.Chinese sentiment analysis using bidirectional LSTM with word embedding[C]//Proceedings of the 2nd International Conference on Cloud Computing and Security.Berlin:Springer,2016:601-610.
[14]PAPPAS N,POPESCU-BELIS A.Multilingual hierarchical at-tention networks for document classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL).2017:1015-1025.
[15]LIU L F,YANG L,ZHANG S W,et al.Convolution neural networks for Chinese microblog sentiment analysis[J].Journal of Chinese Information Processing,2015,29(6):159-165.
[16]ZHANG J,DUAN L G,LI A P,et al.Fine-grained SentimentAnalysis Based on Combination of Attention and Gated Mechanism[J].Computer Science,2021,48(8):226-233.
[17]YUAN H J,ZHANG X,NIU W H,et al.Sentiment AnalysisBased on Multichannel Convolution and Bidirectional GRU with Attention Mechanism[J].Journal of Chinese Information Processing,2019,33(10):109-118.
[18]WANG L Y,LIU C H,CAI D B,et al.Chinese text sentimentanalysis based on character-level two-channel composite network[J].Application Research of Computers,2020,37(9):2674-2678.
[19]MNIH V,HEESS N,GRAVES A.Recurrent models of visual attention[C]//Advances in Neural Information Processing Systems.Montreal,Canada:NPIS,2014:2204-2212.
[20]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[21]CHENG S Y,GUO Z Y,LIU W,et al.Research on Multi-granularity Sentence Interaction Natural Language Inference Based on Attention Mechanism[J].Journal of Chinese Computer Systems,2019,40(6):1215-1220.
[22]CHO K,VAN MERRIENBOER B,GULCEHRE C,et al.Learning phrase representions using RNN encoder-decoder for statistical machine translation [C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing.Stroudsburg,PA:Association for Computational Linguistics,2014:1724-1734.
[23]LI S,ZHAO Z,HU R,et al.Analogical Reasoning on Chinese Morphological and Semantic Relations[C]//Meeting of the Association for Computational Linguistics.2018.
[24]ZAREMBA W,SUTSKEVER I,VINYALS O.Recurrent neural network regularization[J].Eprint Arxiv,2014.
[25]XIAO Z,LIANG P J.Chinese sentiment analysis using bidirectional LSTM with word embedding[C]//Proc of the 2nd International Conference on Cloud Computing and Security.Berlin:Springer,2016:601-610.
[26]GAO Y,GLOWACKA D.Deep gate recurrent neural network[J].arXiv:1604.02910,2016.
[27]WANG Y Q,HUANG M,ZHAO L,et al.Attention-basedLSTM for aspect-level sentiment classification[C]//Procee-dings of Conference on Empirical Methods in Natural Language Precessing.2016:606-615.
[28]WANG Y H,ZHANG C Y,ZHAO B L,et al.Sentiment analysis of twitter data based on CNN[J].Journal of Data Acquisition and Processing,2018,33(5):921-927.
[29]GUO B,ZHANG C X,LIU J M.Improving text classification with weighted word embeddings via a multi-channel TextCNN model[J].Neurocomputing,2019,363:366-374.
[30]CHENG Y,YE Z M,WANG M W,et al.Chinese Text Sentiment Orientation Analysis Based on Convolution Neural Network and Hierarchical Attention Network[J].Journal of Chinese Information Processing,2019,33(1):133-142.
[1] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[2] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[3] ZHAO Dan-dan, HUANG De-gen, MENG Jia-na, DONG Yu, ZHANG Pan. Chinese Entity Relations Classification Based on BERT-GRU-ATT [J]. Computer Science, 2022, 49(6): 319-325.
[4] DING Feng, SUN Xiao. Negative-emotion Opinion Target Extraction Based on Attention and BiLSTM-CRF [J]. Computer Science, 2022, 49(2): 223-230.
[5] YUAN Jing-ling, DING Yuan-yuan, SHENG De-ming, LI Lin. Image-Text Sentiment Analysis Model Based on Visual Aspect Attention [J]. Computer Science, 2022, 49(1): 219-224.
[6] DAI Hong-liang, ZHONG Guo-jin, YOU Zhi-ming , DAI Hong-ming. Public Opinion Sentiment Big Data Analysis Ensemble Method Based on Spark [J]. Computer Science, 2021, 48(9): 118-124.
[7] ZHANG Jin, DUAN Li-guo, LI Ai-ping, HAO Xiao-yan. Fine-grained Sentiment Analysis Based on Combination of Attention and Gated Mechanism [J]. Computer Science, 2021, 48(8): 226-233.
[8] SHI Wei, FU Yue. Microblog Short Text Mining Considering Context:A Method of Sentiment Analysis [J]. Computer Science, 2021, 48(6A): 158-164.
[9] PAN Fang, ZHANG Hui-bing, DONG Jun-chao, SHOU Zhao-yu. Aspect Sentiment Analysis of Chinese Online Course Review Based on Efficient Transformer [J]. Computer Science, 2021, 48(6A): 264-269.
[10] WANG Xi, ZHANG Kai, LI Jun-hui, KONG Fang, ZHANG Yi-tian. Generation of Image Caption of Joint Self-attention and Recurrent Neural Network [J]. Computer Science, 2021, 48(4): 157-163.
[11] LI Jian-lan, PAN Yue, LI Xiao-cong, LIU Zi-wei, WANG Tian-yu. Chinese Commentary Text Research Status and Trend Analysis Based on CiteSpace [J]. Computer Science, 2021, 48(11A): 17-21.
[12] ZHOU Xiao-shi, ZHANG Zi-wei, WEN Juan. Natural Language Steganography Based on Neural Machine Translation [J]. Computer Science, 2021, 48(11A): 557-564.
[13] XU Lin-hong, LIU Xin, YUAN Wei, QI Rui-hua. Construction and Application of Russian Multimodal Emotion Corpus [J]. Computer Science, 2021, 48(11): 312-318.
[14] ZHANG Yan, LI Tian-rui. Review of Comment-oriented Aspect-based Sentiment Analysis [J]. Computer Science, 2020, 47(6): 194-200.
[15] ZHANG Peng-fei, LI Guan-yu, JIA Cai-yan. Truncated Gaussian Distance-based Self-attention Mechanism for Natural Language Inference [J]. Computer Science, 2020, 47(4): 178-183.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.