Computer Science ›› 2018, Vol. 45 ›› Issue (8): 213-217,252.doi: 10.11896/j.issn.1002-137X.2018.08.038

• Artificial Intelligence • Previous Articles     Next Articles

Deeply Hierarchical Bi-directional LSTM for Sentiment Classification

ZENG Zheng1, LI Li2, CHEN Jing3   

  1. College of Journalism and Communication,Chongqing Normal University,Chongqing 401331,China1
    College of Computer and Information Science,Southwest University,Chongqing 400715,China2
    BIM Center,CSDI Engineering Co.,LTD.,Chongqing 401122,China3
  • Received:2018-05-02 Online:2018-08-29 Published:2018-08-29

Abstract: The comments on goods,films and others contribute to assess people’s preference degree for goods,which provides reference for the people who intend to buy the goods,and can help businesses adjust shelves to maximize pro-fits.In recent years,the powerful representation and learning ability in deep learning technologies provides a good support for understanding text semantics and grasping the emotional tendency of texts,especially the long short-term me-mory (LSTM) model in deep learning.The comment is a form of temporal data,which expresses semantic information through the forward arrangement of words.LSTM is a sequential model that reads the comment forward and encodes it into a real vector,and this vector implies the potential semantics of the comment and can be stored and processed by the computer.In this paper,two LSTM models are utilized to read comments from forward and backward directions respectively,and thus the two-way semantic information of the review can be obtained.Then the purpose of obtaining the deep features of comments is achieved by stacking the multilayer bidirectional LSTM.Finally,the model is put into a sentimental classification model to implement the sentiment classification.Experimental results show that the proposed method outperforms baseline LSTM,which means that deeply hierarchical bi-directional LSTM (DHBL) can capture more accurate text information.Compared with the convolutional neural network (CNN) model,the proposed model also achieves better effect.

Key words: LSTM, Deep learning, Sentiment classification

CLC Number: 

  • TP181
[1]IRSOY O,CARDIE C.Opinion Miningwith Deep RecurrentNeural Networks[C]∥Conference on Empirical Methods in Natural Language Processing.2014:720-728.
[2]ZHOU X,WAN X,XIAO J.CMiner:Opinion Extraction andSummarization forChinese Microblogs[J].IEEE Transactions on Knowledge and Data Engineering,2016,28(7):1650-1663.
[3]WEJ J,RUAN H,LI Z.Analysis of economic impact of onlinereviews:an approach for market-driven requirements evolution[M]∥Requirements.Springer Berlin Heidelberg:Engineering,2014:45-59.
[4]SONG G,YE Y,DU X,et al.Short text classification:A survey[J].Journal of Multimedia,2014,9(5):635-643.
[5]WANG B K,HUANG Y F,YANG W X,et al.Short text classification based on strong feature thesaurus[J].Frontiers of Information Technology & Electronic Engineering,2012,13(9):649-659.
[6]KIM K,CHUNG B S,CHOI Y,et al.Language independent semantic kernels for short-text classification[J].Expert Systems with Applications,2014,41(2):735-743.
[7]WANG M,LIN L,WANG F.Improving Short Text Classification through Better Feature Space Selection[C]∥International Conference on Computational Intelligence and Security.IEEE,2014:120-124.
[8]FAN X,HU H.Construction of High-quality Feature Extension Mode Library for Chinese Short-text Classification[C]∥WASE International Conference on Information Engineering.IEEE,2010:87-90.
[9]ZHANG X,WU B.ShortText Classification based on featureextension using The N-Grammodel[C]∥International Confe-rence on Fuzzy Systems & Knowledge Discovery.IEEE,2016:710-716.
[10]HUANG P S,HE X,GAO J,et al.Learning deep structured semantic models for web search using clickthrough data[C]∥Acm International Conference on Conference on Information & Knowledge Management.ACM,2013:2333-2338.
[11]SHEN Y,HE X,GAO J,et al.Alatent semantic model with con-volutional-pooling structure for information retrieval[C]∥International Conference on Conference on Information and Knowledge Management.ACM,2014:101-110.
[12]KIM Y.Convolutional neural networks for sentence classification.arXiv preprint arXiv:1408.5882,2014.
[13]MIKOLOV T,KARAFIÁT M,BURGET L,et al.Recurrentneural network based language model[C]∥INTERSPEECH 2010,Conference of the International Speech Communication Association.DBLP,2010:1045-1048.
[14]MIKOLOV T.Statistical language models based on neural networks[OL]. Accessed on Jan.2018.
[15]WILLIAMS R J,ZIPSER D.Gradient-based learning algorithms for recurrent networks and their computational complexity.Backpropagation:Theory,Architectures,Andapplications,1995,1:433-486.
[16]HOCHREITER S.The Vanishing Gradient Problem DuringLearning Recurrent Neural Nets and Problem Solutions[J].International Journal of Uncertainty,Fuzzinessand Knowledge-Based Systems,1998,6(2):107-116.
[17]GUSTAVSSON A,MAGNUSON A,BLOMBERG B,et al.On the difficulty of training recurrent neural networks[J].Compu-ter Science,2013,52(3):337-345.
[18]HOCHREITER S,SCHMIDHUBER J.Long short-term memory.Neural Computation,1997,9(8):1735-1780.
[19]OLAH C.Understanding LSTMNetworks[OL].
[20]GRAVES A,SCHMIDHUBER J.Framewise phoneme classification with bidirectional LSTM networks[C]∥IEEE International Joint Conference on Neural Networks,2005(IJCNN’05).IEEE,2005:2047-2052.
[21]LIN M,CHEN Q,YAN S.Network In Network[J].arXiv.preprint arXiv?13120.4400,2013.
[22]WU Y,SCHUSTER M,CHEN Z,et al.Google’s Neural Ma-chine Translation System:Bridging the Gap between Human and Machine Translation[J].arXiv preprint arXiv:1609.08144,2016.
[23]HAWKINS D M.The Problem of Overfitting[J].Cheminform,2004,35(19):1-12.
[24]HINTON G E,SRIVASTAVA N,KRIZHEVSKY A,et al.Improving neural networks by preventing co-adaptation of feature detectors[J].Computer Science,2012,3(4):212-223.
[25]MAAS A L,DALY R E,PHAM P T,et al.Learning word vectors for sentiment analysis[C]∥Meeting of the Association for Computational Linguistics:Human Language Technologies.Association for Computational Linguistics,2011:142-150.
[26]NAKOV P,RITTER A,Rosenthal S,et al.SemEval-2016 Task 4:Sentiment Analysis in Twitter[C]∥International Workshop on Semantic Evaluation.2016:1-18.
[27]ANOOP V S,PREM S C.Generating and visualizing topichiera-rchies from microblogs:An iterative latent dirichlet allocation approach[C]∥2015 International Conference on Advances in Computing Communications and Informatics(ICACCI).2015:824-828.
[28]LE Q V,MIKOLOV T.Distributed Representations of Sen-tences and Documents[J].ICML,2014,4(2):1188-1196.
[29]DUCHI J,HAZAN E,SINGER Y.Adaptive subgradient methods for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011,12(7):257-269.
[30]ZEILER M D.Adadelta:an adaptive learning rate method[J].arXiv preprint arXiv:1212.5701,2012.
[31]KINGMA D,BA J.Adam:amethod for stochastic optimization[J].arXiv preprint arXiv:1412.6980,2014.
[1] ZHOU Yan, ZENG Fan-zhi, WU Chen, LUO Yue, LIU Zi-qin. 3D Shape Feature Extraction Method Based on Deep Learning [J]. Computer Science, 2019, 46(9): 47-58.
[2] MA Lu, PEI Wei, ZHU Yong-ying, WANG Chun-li, WANG Peng-qian. Fall Action Recognition Based on Deep Learning [J]. Computer Science, 2019, 46(9): 106-112.
[3] LI Qing-hua, LI Cui-ping, ZHANG Jing, CHEN Hong, WANG Shao-qing. Survey of Compressed Deep Neural Network [J]. Computer Science, 2019, 46(9): 1-14.
[4] WANG Yan-ran, CHEN Qing-liang, WU Jun-jun. Research on Image Semantic Segmentation for Complex Environments [J]. Computer Science, 2019, 46(9): 36-46.
[5] SUN Zhong-feng, WANG Jing. RCNN-BGRU-HN Network Model for Aspect-based Sentiment Analysis [J]. Computer Science, 2019, 46(9): 223-228.
[6] MIAO Yong-wei, LI Gao-yi, BAO Chen, ZHANG Xu-dong, PENG Si-long. Image Localized Style Transfer Based on Convolutional Neural Network [J]. Computer Science, 2019, 46(9): 259-264.
[7] DENG Cun-bin, YU Hui-qun, FAN Gui-sheng. Integrating Dynamic Collaborative Filtering and Deep Learning for Recommendation [J]. Computer Science, 2019, 46(8): 28-34.
[8] DU Wei, DING Shi-fei. Overview on Multi-agent Reinforcement Learning [J]. Computer Science, 2019, 46(8): 1-8.
[9] GUO Xu, ZHU Jing-hua. Deep Neural Network Recommendation Model Based on User Vectorization Representation and Attention Mechanism [J]. Computer Science, 2019, 46(8): 111-115.
[10] ZHANG Yi-jie, LI Pei-feng, ZHU Qiao-ming. Event Temporal Relation Classification Method Based on Self-attention Mechanism [J]. Computer Science, 2019, 46(8): 244-248.
[11] LIU Meng-juan,ZENG Gui-chuan,YUE Wei,QIU Li-zhou,WANG Jia-chang. Review on Click-through Rate Prediction Models for Display Advertising [J]. Computer Science, 2019, 46(7): 38-49.
[12] LI Zhou-jun,WANG Chang-bao. Survey on Deep-learning-based Machine Reading Comprehension [J]. Computer Science, 2019, 46(7): 7-12.
[13] SHEN Chen-lin, ZHANG Lu, WU Liang-qing, LI Shou-shan. Sentiment Classification Towards Question-Answering Based on Bidirectional Attention Mechanism [J]. Computer Science, 2019, 46(7): 151-156.
[14] HAN Hui,WANG Li-ming,CHAI Yu-mei,LIU Zhen. Text Sentiment Classification Based on Deep Forests with Enhanced Features [J]. Computer Science, 2019, 46(7): 172-179.
[15] ZHANG Lin-na,CHEN Jian-qiang,CHEN Xiao-ling,CEN Yi-gang,KAN Shi-chao. Lightweight SSD Network for Real-time Object Detection in Automotive Videos [J]. Computer Science, 2019, 46(7): 233-237.
Full text



[1] . [J]. Computer Science, 2018, 1(1): 1 .
[2] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75, 88 .
[3] XIA Qing-xun and ZHUANG Yi. Remote Attestation Mechanism Based on Locality Principle[J]. Computer Science, 2018, 45(4): 148 -151, 162 .
[4] LI Bai-shen, LI Ling-zhi, SUN Yong and ZHU Yan-qin. Intranet Defense Algorithm Based on Pseudo Boosting Decision Tree[J]. Computer Science, 2018, 45(4): 157 -162 .
[5] WANG Huan, ZHANG Yun-feng and ZHANG Yan. Rapid Decision Method for Repairing Sequence Based on CFDs[J]. Computer Science, 2018, 45(3): 311 -316 .
[6] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[7] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[8] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[9] LIU Qin. Study on Data Quality Based on Constraint in Computer Forensics[J]. Computer Science, 2018, 45(4): 169 -172 .
[10] ZHONG Fei and YANG Bin. License Plate Detection Based on Principal Component Analysis Network[J]. Computer Science, 2018, 45(3): 268 -273 .