Computer Science ›› 2021, Vol. 48 ›› Issue (2): 245-249.doi: 10.11896/jsjkx.200100078

• Artificial Intelligence • Previous Articles     Next Articles

Recurrent Convolution Attention Model for Sentiment Classification

CHEN Qian1,2, CHE Miao-miao1, GUO Xin1, WANG Su-ge1,2   

  1. 1 School of Computer & Information Technology,Shanxi University,Taiyuan 030006,China
    2 Key Laboratory of Computational Intelligence and Chinese Information Processing,Ministry of Education,Taiyuan 030006,China
  • Received:2020-01-13 Revised:2020-06-19 Online:2021-02-15 Published:2021-02-04
  • About author:CHEN Qian,born in 1983,Ph.D,asso-ciate professor,master supervisor,is a member of China Computer Federation.His main research interests include to-pic detection and evolution,machine reading comprehension and natural language processing.
    GUO Xin,born in 1982,Ph.D,associate professor,master supervisor,is a member of China Computer Federation.Her main research interests include feature learning and natural language proces-sing.
  • Supported by:
    The National Natural Science Foundation of China(61502288,61403238),Natural Science Foundation of Shanxi Pro-vince(201901D111032,201701D221101) and Key Research and Development Project of Shanxi Province(201803D421024).

Abstract: Sentiment classification has important application value for downstream applications,including recommendation system,automatic question answering and reading comprehension.It is an important research direction in the field of natural language processing.The task of sentiment classification depends on global and local information hidden in context.However,exis-ting neural network models can not capture the local and global information of context at the same time.In this paper,a recurrent convolutional attention model (LSTM-CNN-ATT,LCA) is proposed for single label and multi-label sentiment classification tasks.It uses attention mechanism to fuse the local information extraction ability of convolutional neural network and the global information extraction ability of recurrent neural network,including word embedding layer,context representation layer,convolution layer and attention layer.For the multi-label sentiment classification task,the topic information is added to the attention layer to further guide the accurate extraction of multi-label emotion tendency.The F1 index on two single label datasets reaches 82.1%,which is equivalent to the frontier single label model.On two multi-label datasets,the experimental results on small datasets are close to the benchmark model,and the F1 index on large datasets reaches 78.38%,which is higher than the state-of-the-art model.It indicates that LCA model has high stability and strong universality.

Key words: Attention mechanism, Convolutional neural network, Recurrent neural network, Sentiment classification

CLC Number: 

  • TP391
[1] CHUNG J,GULCEHRE C,CHO K,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[J].arXiv:1412.3555,2014.
[2] YIN W,KANN K,YU M,et al.Comparative study of cnn and rnn for natural language processing[J].arXiv:1702.01923,2017.
[3] DOMINIK S,ANDREAS M,SVEN B.Evaluation of pooling ope-rations in convolutional architectures for object recognition[C]//International Conference on Artificial Neural Networks.2010:92-101.
[4] RONAN C,JASON W,LEON B,et al.Natural language processing (almost) from scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[5] ZHU J,HASTIE T.Kernel logistic regression and the importvector machine[C]//International Conference on Neural Information Processing Systems:Natural & Synthetic.2001:1081-1088.
[6] CORTES C,VAPNIK V.Support-vector networks[J].Machine Learning,1995,20(3):273-297.
[7] LI Y,WEI B,LIU Y,et al.Incorporating knowledge into neural network for text representation[J].Expert Systems with Applications,2018,96(4):103-114.
[8] KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.2014:1746-1751.
[9] ALEX G.Generating Sequences with Recurrent Neural Net-works[J].arXiv:1308.0850,2013.
[10] BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2016.
[11] TANG D,QIN B,LIU T.Document modeling with gated recurrent neural network for sentiment classification[C]//Procee-dings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:1422-1432.
[12] XU J,CHEN D,QIU X,et al.Cached long short-term memory neural networks for document-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1660-1669.
[13] TIAN Z,RONG W,SHI L,et al.Attention Aware Bidirectional Gated Recurrent Unit Based Framework for Sentiment Analysis[C]//Proceedings of the 2018 Conference of the International Conference on Knowledge Science,Engineering and Management.Springer.Cham,2018:67-78.
[14] DELVIN J,CHANG M W,LEE K,et al.BERT:Pre-Training of Deep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the Annual Confe-rence of the North American Chapter of the Association for Computational Linguistics.2019:4171-4186.
[15] WANG Y,HUANG M,ZHAO L,et al.Attention-based lstm for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:606-615.
[16] CHENG J,ZHAO S,ZHANG J,et al.Aspect-level sentiment classification with HEAT (hierarchical attention) network[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.2017:97-106.
[17] XUE W,LI T.Aspect based sentiment analysis with gated con-volutional networks[C]//Proceedings of the 56th Annual Mee-ting of the Association for Computational Linguistics.2018:2514-2523.
[18] JIANG M,ZHANG W,ZHANG M,et al.An LSTM-CNN attention approach for aspect-level sentiment classification[J].Journal of Computational Methods in Sciences and Engineering,2019,(19):859-868.
[19] QUOC L,TOMAS M.Distributed representations of sentences and documents[C]//International Conference on Machine Learning.2014:1188-1196.
[20] JEFFREY P,RICHARD S,CHRISTOPHER M.Glove:Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing.2014:1532-1543.
[21] KAI S,RICHARD S,CHRISTOPHER M.Improved semantic representations from tree-structured long short-term memory networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing,Association for Computational Linguistics (ACL).2015:1556-1566.
[22] RICHARD S,ALEX P,JEAN W,et al.Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.2013:1631-1642.
[23] ZHANG R,HONGLAK L,DRAGOMIR R.Dependency sensitive convolutional neural networks for modeling sentences and documents[J].arXiv:1611.02361,2016.
[24] ZHANG T,HUANG M,ZHAO L.Learning structured representation for text classification via reinforcement learning[C]//proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence.New Orleans,Louisiana,USA,2018.
[25] YIN W,SCHUTZE H.Multichannel variable-size convolutionfor sentence classification[J].arXiv: 603.04513,2016.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[3] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[4] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[5] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[8] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[9] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[10] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[11] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[12] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[13] XIONG Luo-geng, ZHENG Shang, ZOU Hai-tao, YU Hua-long, GAO Shang. Software Self-admitted Technical Debt Identification with Bidirectional Gate Recurrent Unit and Attention Mechanism [J]. Computer Science, 2022, 49(7): 212-219.
[14] PENG Shuang, WU Jiang-jiang, CHEN Hao, DU Chun, LI Jun. Satellite Onboard Observation Task Planning Based on Attention Neural Network [J]. Computer Science, 2022, 49(7): 242-247.
[15] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!