Computer Science ›› 2019, Vol. 46 ›› Issue (10): 258-264.doi: 10.11896/jsjkx.180901687

• Artificial Intelligence • Previous Articles     Next Articles

Sentiment Analysis Based on Word Embedding Auxiliary Mechanism

HAN Xu-li1, ZENG Bi-qing2, ZENG Feng1, ZHANG Min1, SHANG Qi1   

  1. (School of Computer Science,South China Normal University,Guangzhou 510631,China)1
    (School of Software,South China Normal University,Foshan,Guangdong 528225,China)2
  • Received:2018-09-09 Revised:2018-12-22 Online:2019-10-15 Published:2019-10-21

Abstract: Text sentiment analysis is an important research direction in natural language processing,and how to analyze the sentiment polarity of long text is a research hotspot.At present,majority of the researches tend to apply word embedding in neural network models for sentiment analysis.Although this method has a good representation ability of word features,it has some weaknesses for long text.Extremely long text will bring heavy burden to the model and make it consume more time and resources in training process.In light of this,this paper proposed an attention neural network model based on word embedding auxiliary mechanism,namely WEAN,and it is applied to sentiment analysis for long text.The model deals with some training burdens of long text in neural network model by using word embedding auxi-liary mechanism and utilizes bidirectional recurrent neural network and mechanism to obtain context information.At the same time,it captures information with different importance degree in sequences,thus improving the sentiment classification performance.Experiment was conducted on IMDB,Yelp 2013 and Yelp 2014 datasets.The results show that the accuracy of the sentiment analysis of the proposed model increases by 1.1%,2.0% and 2.6% respectively on three datasets compared with NSC+LA model.

Key words: Sentiment analysis, Word embedding, Attention mechanism, Neural network, Natural language processing

CLC Number: 

  • TP391
[1]WANG S,MANNING C D.Baselines and bigrams:simple,good sentiment and topic classification[C]//Meeting of the Association for Computational Linguistics:Short Papers.Jeju Island,Korea:ACL 2012:90-94.
[2]BENGIO Y.Learning Deep Architectures for AI [J].Founda-tions & Trends® in Machine Learning,2009,2(1):1-127.
[3]SOCHER R,LIN C,MANNING C D,et al.Parsing Natural Scenes and Natural Language with Recursive Neural Networks[C]//International Conference on Machine Learning.Bellevue,Washington:ICML,2011:129-136.
[4]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013.
[5]HOCHREITER S,SCHMIDHUBER J.Long short-term memory [J].Neural Computation,1997,9(8):1735-1780.
[6]KIM Y.Convolutional Neural Networks for Sentence Classification[J].Empirical Methods in Natural Language Processing,2014,1:1746-1751.
[7]ZHANG Y,GAN Z,FAN K,et al.Adversarial Feature Matching for Text Generation[C]//International Conference on Machine Learning.Sydney:ICAL,2017:4006-4015.
[8]GRAVES A,JAITLY N,MOHAMED A,et al.Hybrid speech recognition with Deep Bidirectional LSTM[C]//IEEE Automa-tic Speech Recognition and Understanding Workshop.Olomouc,Czech Republic:IEEE,2013:273-278.
[9]YANG Z,YANG D,DYER C,et al.Hierarchical Attention Networks for Document Classification[C]//Northamerican Chapter of the Association for Computational Linguistics.San Diego,California:ACL,2016:1480-1489.
[10]PANG B,LEE L,VAITHYANATHAN S,et al.Thumbs up? Sentiment Classification using Machine Learning Techniques[J].Empirical Methods in Natural Language Processing,2002,1(1):79-86.
[11]TURNEY P D.Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews[J].Meeting of the Association for Computational Linguistics,2002,10:417-424.
[12]CONNEAU A,SCHWENK H,BARRAULT L,et al.Very deep convolutional networks for text classification[J].Conference of the European Chapter of the Association for Computational Linguistics,2017,1:1107-1116.
[13]SHEN D,MIN M R,LI Y,et al.Adaptive convolutional filter generation for natural language understanding[J].arXiv:2017,1709.08294.
[14]SHI B,FU Z,BING L,et al.Learning Domain-Sensitive and Sentiment-Aware Word Embeddings[J].Meeting of the Association for Computational Linguistics,2018,1(1):2494-2504.
[15]ADI Y,KERMANY E,BELINKOV Y,et al.Fine-grained Ana-lysis of Sentence Embeddings Using Auxiliary Prediction Tasks[J/OL].International Conference on Learning Representations,2017.
[16]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P,et al.A Convolutional Neural Network for Modelling Sentences[J].Meeting of the Association for Computational Linguistics,2014,1:655-665.
[17]JOHNSON R,ZHANG T.Effective Use of Word Order for Text Categorization with Convolutional Neural Networks[OL].
[18]ZHANG X,ZHAO J,LECUN Y.Character-level c-onvolutional networks for text classification [C]//Advances in Neural Information Processing Systems.Montreal,Canada:NPIS,2015:649-657.
[19]LI J,LUONG T,JURAFSKY D,et al.When Are Tree Struc-tures Necessary for Deep Learning of Representations[J].Empiricalmethods in Natural Language Processing,2015,1(1):2304-2314.
[20]LE Q V,MIKOLOV T.Distributed Representations of Sen-tences and Documents[OL].
[21]TANG D,QIN B,LIU T.Document Modeling with Gated Recurrent Neural Network for Sentiment Classification[C]//Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal:ACL,2015:1422-1432.
[22]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is All you Need[C]//Neural Information Processing Systems.Long Beach:NIPS,2017:5998-6008.
[23]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTM for Aspect-level Sentiment Classification[C]//Conference on Empirical Methods in Natural Language Processing.Austin,Texas:ACL,2017:606-615.
[24]MNIH V,HEESS N,GRAVES A.Recurrent models of visual attention[C]//Advances in Neural Information Processing Systems.Montreal,Canada:NPIS,2014:2204-2212.
[25]BAHDANAU D,CHO K,BENGIO Y,et al.Neural Machine Translation by Jointly Learning to Align and Translate[OL].
[26]LUONG T,PHAM H,MANNING C D,et al.Effective Approaches to Attention-based Neural Machine Translation[J].Empi-rical Methods in Natural Language Processing,2015,1:1412-1421.
[27]FAN R E,CHANG K W,HSIEH C J,et al.LIBLINEAR:A Library for Large Linear Classification [J].Journal of Machine Learning Research,2008,9(9):1871-1874.
[28]KIRITCHENKO S,ZHU X,MOHAMMAD S M,et al.Senti-ment analysis of short informal texts[J].Journal of Artificial Intelligence Research,2014,50(1):723-762.
[29]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed Representations of Words and Phrases and their compositionality[J].Advances in Neural Information Processing Systems,2013,26:3111-3119.
[30]TANG D,WEI F,YANG N,et al.Learning sentiment-specific word embedding for twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.Baltimore,Maryland,USA:ACL,2014,1:1555-1565.
[31]SOCHER R,PERELYGIN A,WU J,et al.Recursive deep mo-dels for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.Seattle,Washington,USA:ACL.2013:1631-1642.
[32]LE Q,MIKOLOV T.Distributed representations of sentences and documents[C]//International Conference on Machine Learning.Beijing,China:ICAL,2014:1188-1196.
[1] YU Xue-yong, CHEN Tao. Privacy Protection Offloading Algorithm Based on Virtual Mapping in Edge Computing Scene [J]. Computer Science, 2021, 48(1): 65-71.
[2] SHAN Mei-jing, QIN Long-fei, ZHANG Hui-bing. L-YOLO:Real Time Traffic Sign Detection Model for Vehicle Edge Computing [J]. Computer Science, 2021, 48(1): 89-95.
[3] HE Yan-hui, WU Gui-xing, WU Zhi-qiang. Domain Alignment Based Object Detection of X-ray Images [J]. Computer Science, 2021, 48(1): 175-181.
[4] ZHAO Jia-qi, WANG Han-zheng, ZHOU Yong, ZHANG Di, ZHOU Zi-yuan. Remote Sensing Image Description Generation Method Based on Attention and Multi-scale Feature Enhancement [J]. Computer Science, 2021, 48(1): 190-196.
[5] LIU Yang, JIN Zhong. Fine-grained Image Recognition Method Combining with Non-local and Multi-region Attention Mechanism [J]. Computer Science, 2021, 48(1): 197-203.
[6] LI Ya-nan, HU Yu-jia, GAN Wei, ZHU Min. Survey on Target Site Prediction of Human miRNA Based on Deep Learning [J]. Computer Science, 2021, 48(1): 209-216.
[7] WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM [J]. Computer Science, 2021, 48(1): 226-232.
[8] ZHANG Yu-shuai, ZHAO Huan, LI Bo. Semantic Slot Filling Based on BERT and BiLSTM [J]. Computer Science, 2021, 48(1): 247-252.
[9] TONG Xin, WANG Bin-jun, WANG Run-zheng, PAN Xiao-qin. Survey on Adversarial Sample of Deep Learning Towards Natural Language Processing [J]. Computer Science, 2021, 48(1): 258-267.
[10] ZHANG Yan-mei, LOU Yin-cheng. Deep Neural Network Based Ponzi Scheme Contract Detection Method [J]. Computer Science, 2021, 48(1): 273-279.
[11] WANG Run-zheng, GAO Jian, HUANG Shu-hua, TONG Xin. Malicious Code Family Detection Method Based on Knowledge Distillation [J]. Computer Science, 2021, 48(1): 280-286.
[12] LU Long-long, CHEN Tong, PAN Min-xue, ZHANG Tian. CodeSearcher:Code Query Using Functional Descriptions in Natural Languages [J]. Computer Science, 2020, 47(9): 1-9.
[13] TIAN Ye, SHOU Li-dan, CHEN Ke, LUO Xin-yuan, CHEN Gang. Natural Language Interface for Databases with Content-based Table Column Embeddings [J]. Computer Science, 2020, 47(9): 60-66.
[14] ZHUANG Shi-jie, YU Zhi-yong, GUO Wen-zhong, HUANG Fang-wan. Short Term Load Forecasting via Zoneout-based Multi-time Scale Recurrent Neural Network [J]. Computer Science, 2020, 47(9): 105-109.
[15] ZHANG Jia-jia, ZHANG Xiao-hong. Multi-branch Convolutional Neural Network for Lung Nodule Classification and Its Interpretability [J]. Computer Science, 2020, 47(9): 129-134.
Full text



[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8] GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[9] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[10] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .