Computer Science ›› 2018, Vol. 45 ›› Issue (8): 213-217.doi: 10.11896/j.issn.1002-137X.2018.08.038

• Artificial Intelligence • Previous Articles     Next Articles

Deeply Hierarchical Bi-directional LSTM for Sentiment Classification

ZENG Zheng1, LI Li2, CHEN Jing3   

  1. College of Journalism and Communication,Chongqing Normal University,Chongqing 401331,China1
    College of Computer and Information Science,Southwest University,Chongqing 400715,China2
    BIM Center,CSDI Engineering Co.,LTD.,Chongqing 401122,China3
  • Received:2018-05-02 Online:2018-08-29 Published:2018-08-29

Abstract: The comments on goods,films and others contribute to assess people’s preference degree for goods,which provides reference for the people who intend to buy the goods,and can help businesses adjust shelves to maximize pro-fits.In recent years,the powerful representation and learning ability in deep learning technologies provides a good support for understanding text semantics and grasping the emotional tendency of texts,especially the long short-term me-mory (LSTM) model in deep learning.The comment is a form of temporal data,which expresses semantic information through the forward arrangement of words.LSTM is a sequential model that reads the comment forward and encodes it into a real vector,and this vector implies the potential semantics of the comment and can be stored and processed by the computer.In this paper,two LSTM models are utilized to read comments from forward and backward directions respectively,and thus the two-way semantic information of the review can be obtained.Then the purpose of obtaining the deep features of comments is achieved by stacking the multilayer bidirectional LSTM.Finally,the model is put into a sentimental classification model to implement the sentiment classification.Experimental results show that the proposed method outperforms baseline LSTM,which means that deeply hierarchical bi-directional LSTM (DHBL) can capture more accurate text information.Compared with the convolutional neural network (CNN) model,the proposed model also achieves better effect.

Key words: Deep learning, LSTM, Sentiment classification

CLC Number: 

  • TP181
[1]IRSOY O,CARDIE C.Opinion Miningwith Deep RecurrentNeural Networks[C]∥Conference on Empirical Methods in Natural Language Processing.2014:720-728.
[2]ZHOU X,WAN X,XIAO J.CMiner:Opinion Extraction andSummarization forChinese Microblogs[J].IEEE Transactions on Knowledge and Data Engineering,2016,28(7):1650-1663.
[3]WEJ J,RUAN H,LI Z.Analysis of economic impact of onlinereviews:an approach for market-driven requirements evolution[M]∥Requirements.Springer Berlin Heidelberg:Engineering,2014:45-59.
[4]SONG G,YE Y,DU X,et al.Short text classification:A survey[J].Journal of Multimedia,2014,9(5):635-643.
[5]WANG B K,HUANG Y F,YANG W X,et al.Short text classification based on strong feature thesaurus[J].Frontiers of Information Technology & Electronic Engineering,2012,13(9):649-659.
[6]KIM K,CHUNG B S,CHOI Y,et al.Language independent semantic kernels for short-text classification[J].Expert Systems with Applications,2014,41(2):735-743.
[7]WANG M,LIN L,WANG F.Improving Short Text Classification through Better Feature Space Selection[C]∥International Conference on Computational Intelligence and Security.IEEE,2014:120-124.
[8]FAN X,HU H.Construction of High-quality Feature Extension Mode Library for Chinese Short-text Classification[C]∥WASE International Conference on Information Engineering.IEEE,2010:87-90.
[9]ZHANG X,WU B.ShortText Classification based on featureextension using The N-Grammodel[C]∥International Confe-rence on Fuzzy Systems & Knowledge Discovery.IEEE,2016:710-716.
[10]HUANG P S,HE X,GAO J,et al.Learning deep structured semantic models for web search using clickthrough data[C]∥Acm International Conference on Conference on Information & Knowledge Management.ACM,2013:2333-2338.
[11]SHEN Y,HE X,GAO J,et al.Alatent semantic model with con-volutional-pooling structure for information retrieval[C]∥International Conference on Conference on Information and Knowledge Management.ACM,2014:101-110.
[12]KIM Y.Convolutional neural networks for sentence classification.arXiv preprint arXiv:1408.5882,2014.
[13]MIKOLOV T,KARAFIÁT M,BURGET L,et al.Recurrentneural network based language model[C]∥INTERSPEECH 2010,Conference of the International Speech Communication Association.DBLP,2010:1045-1048.
[14]MIKOLOV T.Statistical language models based on neural networks[OL].https://pdfs.semanticscholar.org/e753/714f98099e3da1e96c652d34cc45e315ad23.pdf. Accessed on Jan.2018.
[15]WILLIAMS R J,ZIPSER D.Gradient-based learning algorithms for recurrent networks and their computational complexity.Backpropagation:Theory,Architectures,Andapplications,1995,1:433-486.
[16]HOCHREITER S.The Vanishing Gradient Problem DuringLearning Recurrent Neural Nets and Problem Solutions[J].International Journal of Uncertainty,Fuzzinessand Knowledge-Based Systems,1998,6(2):107-116.
[17]GUSTAVSSON A,MAGNUSON A,BLOMBERG B,et al.On the difficulty of training recurrent neural networks[J].Compu-ter Science,2013,52(3):337-345.
[18]HOCHREITER S,SCHMIDHUBER J.Long short-term memory.Neural Computation,1997,9(8):1735-1780.
[19]OLAH C.Understanding LSTMNetworks[OL].http://colah.github.io/posts/2015-08-Understanding-LSTMs.
[20]GRAVES A,SCHMIDHUBER J.Framewise phoneme classification with bidirectional LSTM networks[C]∥IEEE International Joint Conference on Neural Networks,2005(IJCNN’05).IEEE,2005:2047-2052.
[21]LIN M,CHEN Q,YAN S.Network In Network[J].arXiv.preprint arXiv?13120.4400,2013.
[22]WU Y,SCHUSTER M,CHEN Z,et al.Google’s Neural Ma-chine Translation System:Bridging the Gap between Human and Machine Translation[J].arXiv preprint arXiv:1609.08144,2016.
[23]HAWKINS D M.The Problem of Overfitting[J].Cheminform,2004,35(19):1-12.
[24]HINTON G E,SRIVASTAVA N,KRIZHEVSKY A,et al.Improving neural networks by preventing co-adaptation of feature detectors[J].Computer Science,2012,3(4):212-223.
[25]MAAS A L,DALY R E,PHAM P T,et al.Learning word vectors for sentiment analysis[C]∥Meeting of the Association for Computational Linguistics:Human Language Technologies.Association for Computational Linguistics,2011:142-150.
[26]NAKOV P,RITTER A,Rosenthal S,et al.SemEval-2016 Task 4:Sentiment Analysis in Twitter[C]∥International Workshop on Semantic Evaluation.2016:1-18.
[27]ANOOP V S,PREM S C.Generating and visualizing topichiera-rchies from microblogs:An iterative latent dirichlet allocation approach[C]∥2015 International Conference on Advances in Computing Communications and Informatics(ICACCI).2015:824-828.
[28]LE Q V,MIKOLOV T.Distributed Representations of Sen-tences and Documents[J].ICML,2014,4(2):1188-1196.
[29]DUCHI J,HAZAN E,SINGER Y.Adaptive subgradient methods for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011,12(7):257-269.
[30]ZEILER M D.Adadelta:an adaptive learning rate method[J].arXiv preprint arXiv:1212.5701,2012.
[31]KINGMA D,BA J.Adam:amethod for stochastic optimization[J].arXiv preprint arXiv:1412.6980,2014.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[9] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[10] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[11] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[12] ZHANG Yuan, KANG Le, GONG Zhao-hui, ZHANG Zhi-hong. Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM [J]. Computer Science, 2022, 49(7): 31-39.
[13] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[14] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[15] ZHU Wen-tao, LAN Xian-chao, LUO Huan-lin, YUE Bing, WANG Yang. Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN [J]. Computer Science, 2022, 49(6A): 378-383.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!