Computer Science ›› 2020, Vol. 47 ›› Issue (10): 222-227.doi: 10.11896/jsjkx.190900173

• Artificial Intelligence • Previous Articles     Next Articles

Comment Sentiment Classification Using Cross-attention Mechanism and News Content

WANG Qi-fa, WANG Zhong-qing, LI Shou-shan, ZHOU Guo-dong   

  1. School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215006,China
  • Received:2019-09-25 Revised:2019-12-06 Online:2020-10-15 Published:2020-10-16
  • About author:WANG Qi-fa,born in 1994,postgra-duate,is a member of China Computer Federation.His main research interests include natural language processing and emotion analysis.
    WANG Zhong-qing,born in 1987,Ph.D,postgraduate supervisor,is a member of China Computer Federation.His main research interests include natural language understanding and information extraction.
  • Supported by:
    Young Scientists Fund of the National Natural Science Foundation of China (61806137,61702518) and Natural Science Foundation of the Jiangsu Higher Education Institutions of China (18KJB520043)

Abstract: At present,news comment has become important news derived data.News comment expresses commentators’ views,positions and personal feelings on news events.Through the analysis of sentiment orientation of news comment,it is helpful to understand the social public opinion and trend.Therefore,the sentiment research of news comment is favored by many scholars.The usual news comment sentiment analysis only considers the information of the comment text itself.However,news comment text information is often closely related to news content information.Based on this,this paper proposed a comment sentiment classification method using cross-attention mechanism and combined with news content.Firstly,the bi-directional long short-term memory network model is used to characterize the news content and the comment text respectively.Then,the cross-attention mechanism is used to further capture important information,and obtain the vector representation of the two updated news content texts and comment texts.And then the semantic representation obtained by splicing them together is input into the full connection layer,and sigmoid activation function is used for classification prediction,so as to realize the sentiment classification of news comments.The results show that the model of comment sentiment classification using cross-attention mechanism and news content can effectively improve the accuracy of sentiment classification of news comment,and this model improves by 1.72%,3.24% and 6.21% on F1 respectively compare with the three benchmark models.

Key words: Attention mechanism, Bi-directional Long Short-Term Memory Network, Sentiment Classification

CLC Number: 

  • TP391.1
[1]FAN W,SUN S.Sentiment classification for online comments on Chinese news[C]//International Conference on Computer Appli-cation & System Modeling.IEEE,2010.
[2]NING Y,ZHU T,WANG Y.Affective-word based Chinese text sentiment classification[C]//International Conference on Pervasive Computing & Applications.IEEE,2011.
[3]SU Z,HUA X,ZHANG D,et al.Chinese sentiment classification using a neural network tool-Word2vec[C]//International Conference on Multisensor Fusion & Information Integration for Intelligent Systems.2014.
[4]ZHANG D,XU H,SU Z,et al.Chinese comments sentimentclassification based on word2vec and SVMperf[J].Expert Systems with Applications,2015,42(4):1857-1863.
[5]ABDULLIN Y B,IVANOV V V.Deep learning model for bilingual sentiment classification of short texts[J].Naucˇno-tehnicˇeskijVestnik Informacionnyh Tehnologij,2017,17(1):129-136.
[6]ZHOU S,CHEN Q,WANG X.Active deep networks for semi-supervised sentiment classification[C]//Proceedings of the 23rd International Conference on Computational Linguistics:Posters.Association for Computational Linguistics,2010:1515-1523.
[7]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[8]KIM Y.Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 conference on empirical methods in natural language processing.2014:1746-1751.
[9]CAO Y,XU R,CHEN T.Combining Convolutional Neural Network and Support Vector Machine for Sentiment Classification[C]//Proceedings of the Meeting of the Chinese National Conference on Social Media Processing.2015:144-155.
[10]TANG D,QIN B,LIU T.Document modeling with gated recurrent neural network for sentiment classification[C]//Proceedings of the 2015 conference on empirical methods in natural language processing.2015:1422-1432.
[11]HOCHREITER S,SCHMIDHUBER,JÜRGEN.Long Short-Term Memory[J].Neural Computation,1997,9(8):1735-1780.
[12]WANG X,LIU Y,CHENGJIE S U N,et al.Predicting polarities of tweets by composing word embeddings with long short-term memory[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1:Long Papers).2015,1:1343-1353.
[13]TAI K S,SOCHER R,MANNING C D.Improved SemanticRepresentations From Tree-Structured Long Short-Term Memory Networks [J].Computer Science,2015,5(1):36.
[14]MNIH V,HEESS N,GRAVES A.Recurrent models of visual attention [J].Advances in Neural Information Processing Systems,2014(3):2204-2212.
[15]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate [C]//Proceedings of the ICLR.2015:1-15.
[16]LUONG M,MANNING C.D.Effective Approaches to Attention-based Neural Machine Translation [C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:1412-1421.
[17]YANG Z,YANG D,DYER C,et al.Hierarchical Attention Networks for Document Classification[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2017:1480-1489.
[18]LI Z,WEI Y,ZHANG Y,et al.Hierarchical attention transfer network for cross-domain sentiment classification[C]//Thirty-Second AAAI Conference on Artificial Intelligence.2018.
[19]ZHOU X,WAN X,XIAO J.Attention-based LSTM network for cross-lingual sentiment classification[C]//Proceedings of the 2016 conference on empirical methods in natural language processing.2016:247-256
[1] ZHAO Jia-qi, WANG Han-zheng, ZHOU Yong, ZHANG Di, ZHOU Zi-yuan. Remote Sensing Image Description Generation Method Based on Attention and Multi-scale Feature Enhancement [J]. Computer Science, 2021, 48(1): 190-196.
[2] LIU Yang, JIN Zhong. Fine-grained Image Recognition Method Combining with Non-local and Multi-region Attention Mechanism [J]. Computer Science, 2021, 48(1): 197-203.
[3] WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM [J]. Computer Science, 2021, 48(1): 226-232.
[4] WANG Run-zheng, GAO Jian, HUANG Shu-hua, TONG Xin. Malicious Code Family Detection Method Based on Knowledge Distillation [J]. Computer Science, 2021, 48(1): 280-286.
[5] PAN Zu-jiang, LIU Ning, ZHANG Wei, WANG Jian-yong. MTHAM:Multitask Disease Progression Modeling Based on Hierarchical Attention Mechanism [J]. Computer Science, 2020, 47(9): 185-189.
[6] ZHAO Wei, LIN Yu-ming, WANG Chao-qiang, CAI Guo-yong. Opinion Word-pairs Collaborative Extraction Based on Dependency Relation Analysis [J]. Computer Science, 2020, 47(8): 164-170.
[7] YUAN Ye, HE Xiao-ge, ZHU Ding-kun, WANG Fu-lee, XIE Hao-ran, WANG Jun, WEI Ming-qiang, GUO Yan-wen. Survey of Visual Image Saliency Detection [J]. Computer Science, 2020, 47(7): 84-91.
[8] LIU Yan, WEN Jing. Complex Scene Text Detection Based on Attention Mechanism [J]. Computer Science, 2020, 47(7): 135-140.
[9] YU Yi-lin, TIAN Hong-tao, GAO Jian-wei and WAN Huai-yu. Relation Extraction Method Combining Encyclopedia Knowledge and Sentence Semantic Features [J]. Computer Science, 2020, 47(6A): 40-44.
[10] NI Hai-qing, LIU Dan, SHI Meng-yu. Chinese Short Text Summarization Generation Model Based on Semantic-aware [J]. Computer Science, 2020, 47(6): 74-78.
[11] HUANG Yong-tao, YAN Hua. Scene Graph Generation Model Combining Attention Mechanism and Feature Fusion [J]. Computer Science, 2020, 47(6): 133-137.
[12] ZHANG Zhi-yang, ZHANG Feng-li, CHEN Xue-qin, WANG Rui-jin. Information Cascade Prediction Model Based on Hierarchical Attention [J]. Computer Science, 2020, 47(6): 201-209.
[13] DENG Yi-jiao, ZHANG Feng-li, CHEN Xue-qin, AI Qing, YU Su-zhe. Collaborative Attention Network Model for Cross-modal Retrieval [J]. Computer Science, 2020, 47(4): 54-59.
[14] ZHANG Peng-fei, LI Guan-yu, JIA Cai-yan. Truncated Gaussian Distance-based Self-attention Mechanism for Natural Language Inference [J]. Computer Science, 2020, 47(4): 178-183.
[15] YU Shan-shan, SU Jin-dian, LI Peng-fei. Sentiment Classification Method for Sentences via Self-attention [J]. Computer Science, 2020, 47(4): 204-210.
Full text



[1] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[2] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[3] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[4] ZHENG Xiu-lin, SONG Hai-yan and FU Yi-peng. Distinguishing Attack of MORUS-1280-128[J]. Computer Science, 2018, 45(4): 152 -156 .
[5] LI Bai-shen, LI Ling-zhi, SUN Yong and ZHU Yan-qin. Intranet Defense Algorithm Based on Pseudo Boosting Decision Tree[J]. Computer Science, 2018, 45(4): 157 -162 .
[6] WU Shu, ZHOU An-min and ZUO Zheng. PDiOS:Private API Call Detection in iOS Applications[J]. Computer Science, 2018, 45(4): 163 -168 .
[7] ZHANG Jing and ZHU Guo-bin. Hot Topic Discovery Research of Stack Overflow Programming Website Based on CBOW-LDA Topic Model[J]. Computer Science, 2018, 45(4): 208 -214 .
[8] WEN Jun-hao, SUN Guang-hui and LI Shun. Study on Matrix Factorization Recommendation Algorithm Based on User Clustering and Mobile Context[J]. Computer Science, 2018, 45(4): 215 -219 .
[9] JIA Wei, HUA Qing-yi, ZHANG Min-jun, CHEN Rui, JI Xiang and WANG Bo. Mobile Interface Pattern Clustering Algorithm Based on Improved Particle Swarm Optimization[J]. Computer Science, 2018, 45(4): 220 -226 .
[10] DING Shu-yang, LI Bing and SHI Hong-bo. Study on Flexible Job-shop Scheduling Problem Based on Improved Discrete Particle Swarm Optimization Algorithm[J]. Computer Science, 2018, 45(4): 233 -239 .