计算机科学 ›› 2023, Vol. 50 ›› Issue (5): 248-254.doi: 10.11896/jsjkx.220400069

• 人工智能 • 上一篇    下一篇

基于多级多尺度特征提取的CNN-BiLSTM模型的中文情感分析

汪林, 蒙祖强, 杨丽娜   

  1. 广西大学计算机与电子信息学院 南宁 530004
  • 收稿日期:2022-04-07 修回日期:2022-09-22 出版日期:2023-05-15 发布日期:2023-05-06
  • 通讯作者: 蒙祖强(zqmeng@126.com)
  • 作者简介:(wanglingxun2021@163.com)
  • 基金资助:
    国家自然科学基金(62266004,61862005)

Chinese Sentiment Analysis Based on CNN-BiLSTM Model of Multi-level and Multi-scale Feature Extraction

WANG Lin, MENG Zuqiang, YANG Lina   

  1. School of Computer and Electronic Information,Guangxi University,Nanning 530004,China
  • Received:2022-04-07 Revised:2022-09-22 Online:2023-05-15 Published:2023-05-06
  • About author:WANG Lin,born in 1996,postgraduate,is a member of China Computer Federation.His main research interests include natural language processing and machine learning.
    MENG Zuqiang,born in 1974,Ph.D,professor,is a senior member of China Computer Federation.His main research interests include artificial intelligence,multimodal learning and granular computing.
  • Supported by:
    National Natural Science Foundation of China(62266004,61862005).

摘要: 情感分析作为自然语言处理(NLP)的一个研究子领域,在舆情监测方面起着非常重要的作用。在中文情感分析任务中,已有方法仅从单极、单尺度来考虑情感特征,无法充分挖掘和利用情感特征信息,模型性能不理想。针对这一问题,提出了一种多级多尺度特征提取的CNN-BiLSTM模型。该模型首先利用预训练好的中文词向量模型并结合嵌入层微调来获取词级特征;然后利用多尺度短语级特征表征模块和句子级特征表征模块来分别获取短语级和句子级特征,在多尺度短语级特征表征模块中,使用具有不同卷积核尺寸的卷积网络来获取不同尺度的短语级特征;最后使用多级特征融合方法将词级特征、不同尺度的短语级特征以及句子级特征进行融合形成多级联合特征,与单极、单尺度特征相比,多级联合特征具有更多的情感信息。在实验中,使用Accuracy,Precision,Recall,F1这4个评估指标对模型性能进行评估,并与包括支持向量机(SVM)在内的8种方法进行比较。实验结果表明,所提方法在4个评估指标中的得分均优于8种对比方法,证明了所提模型在多级和多尺度特征提取上的优势。

关键词: 自然语言处理, 中文情感分析, 多级多尺度特征, 卷积神经网络, 双向长短期记忆网络

Abstract: Sentiment analysis,as a sub-field of natural language processing(NLP),plays a very important role in public opinion monitoring.In the Chinese sentiment analysis task,the existing methods only consider sentiment features from single-level and single-scale,which cannot fully mine and utilize the sentiment feature information,and the performance of the model is not ideal.To solve this problem,a CNN-BiLSTM model with multi-level and multi-scale feature extraction is proposed.This model first uses a pre-trained Chinese word vector model combined with embedding layer fine-tuning to obtain word-level features.Then,phrase-level and sentence-level features are obtained by multi-scale phrase-level feature representation module and sentence-level feature representation module respectively.In the multi-scale phrase-level feature representation module,convolutional networks with different convolution kernel sizes are used to obtain phrase-level features of different scales.Finally,a multi-level feature fusion method is used to fuse word-level features,phrase-level features of different scales,and sentence-level features to form multi-level joint features.Compared with single-level and single-scale features,multi-level joint features have more sentiment information.In the experiment,four evaluation indicators(Accuracy,Precision,Recall,F1) are used to evaluate the performance of the model and compared with eight methods including support vector machines(SVM).Experimental results show that the proposed method outperforms the eight comparison methods in the four evaluation indicators,which proves the advantages of the proposed model in multi-level and multi-scale feature extraction.

Key words: Natural language processing, Chinese sentiment analysis, Multi-level and multi-scale feature, Convolutional neural network, Bidirectional long short-term memory network

中图分类号: 

  • TP391
[1]WANG L,NIU J W,YU S.Combining Textual Information and Sentiment Diffusion Patterns for Twitter Sentiment Analysis[J].IEEE Transactions on Knowledge and Data Engineering,2020,32(10):2026-2039.
[2]HASSONAH M R,AL-SAYYED R,RODAN A,et al.An effi-cient hybrid filter and evolutionary wrapper approach for sentiment analysis of various topics on Twitter[J].Knowledge-Based Systems,2020,192:1-19.
[3]KAGAN V,STEVENS A,SUBRAHMANIAN V S.UsingTwitter Sentiment to Forecast the 2013 Pakistani Election and the 2014 Indian Election[J].IEEE Intelligent Systems,2015,30(1):2-5.
[4]BACCIANELLA S,ESULI A,SEBASTIANI F.SENTIWORDNET 3.0:An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining[C]//Proceedings of the 7th Interna-tional Conference on Language Resources and Evaluation.Valletta:ELRA,2010:55-57.
[5]DENG S Y,SINHA A P,ZHAO H M.Adapting sentiment lexi-cons to domain-specific social media texts[J].Decision Support Systems,2017,94:65-76.
[6]ASGHAR M Z,KHAN A,AHMAD S,et al.Lexicon-enhanced sentiment analysis framework using rule-based classification scheme[J].Plos One,2017,12(2):1-22.
[7]HAN H Y,ZHANG J P,YANG J,et al.Generate domain-specific sentiment lexicon for review sentiment analysis[J].Multimedia Tools and Applications,2018,77(16):21265-21280.
[8]CAI Y,YANG K,HUANG D P,et al.A hybrid model for opi-nion mining based on domain sentiment dictionary[J].International Journal of Machine Learning and Cybernetics,2019,10(8):2131-2142.
[9]HAJEK P.Combining bag-of-words and sentiment features of annual reports to predict abnormal stock returns[J].Neural Computing & Applications,2018,29(7):343-358.
[10]DEY A,JENAMANI M,THAKKAR J J.Lexical TF-IDF:An n-gram Feature Space for Cross-Domain Classification of Sentiment Reviews[C]//Proceedings of the 7th International Confe-rence on Pattern Recognition and Machine Intelligence.Switzerland:Springer International Publishing,2017:380-386.
[11]MANEK A S,SHENOY P D,MOHAN M C,et al.Aspect term extraction for sentiment analysis in large movie reviews using Gini Index feature selection method and SVM classifier[J].World Wide Web-Internet and Web Information Systems,2017,20(2):135-154.
[12]ROUT J K,CHOO K K R,DASH A K,et al.A model for sentiment and emotion analysis of unstructured social media text[J].Electronic Commerce Research,2018,18(1):181-199.
[13]HUQ M R,ALI A,RAHMAN A.Sentiment Analysis on Twitter Data using KNN and SVM[J].International Journal of Advanced Computer Science and Applications,2017,8(6):19-25.
[14]LI Q,LI P F,MAO K Z,et al.Improving convolutional neural network for text classification by recursive data pruning[J].Neurocomputing,2020,414:143-152.
[15]JELODAR H,WANG Y L,ORJI R,et al.Deep Sentiment Classification and Topic Discovery on Novel Coronavirus or COVID-19 Online Discussions:NLP Using LSTM Recurrent Neural Network Approach[J].IEEE Journal of Biomedical and Health Informatics,2020,24(10):2733-2742.
[16]GAN C Q,WANG L,ZHANG Z F,et al.Sparse attention based separable dilated convolutional neural network for targeted sentiment analysis[J].Knowledge-Based Systems,2020,188:1-10.
[17]ZHAO P L,HOU L L,WU O.Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification[J].Knowledge-Based Systems,2020,193:1-10.
[18]ZHOU J,HUANG J X,HU Q V,et al.SK-GCN:Modeling Syntax and Knowledge via Graph Convolutional Network for aspect-level sentiment classification[J].Knowledge-Based Systems,2020,205:1-10.
[19]HASSAN J,SHOAIB U.Multi-class Review Rating Classification using Deep Recurrent Neural Network[J].Neural Proces-sing Letters,2020,51(1):1031-1048.
[20]AHMED M,CHEN Q,LI Z H.Constructing domain-dependent sentiment dictionary for sentiment analysis[J].Neural Computing & Applications,2020,32(18):14719-14732.
[21]WEI J Y,LIAO J,YANG Z F,et al.BiLSTM with Multi-Polarity Orthogonal Attention for Implicit Sentiment Analysis[J].Neurocomputing,2020,383:165-173.
[22]ZHANG Y,WANG J,ZHANG X.Conciseness is better:Recurrent attention LSTM model for document-level sentiment analysis[J].Neurocomputing,2021,462:101-112.
[23]WANG B,SHAN D,FAN A,et al.A Sentiment Classification Method of Web Social Media Based on Multidimensional and Multilevel Modeling[J].IEEE Transactions on Industrial Informatics,2021,18(2):1240-1249.
[24]LI W,ZHU Y,SHI Y,et al.User reviews:Sentiment analysisusing lexicon integrated two-channel CNN-LSTM family models[J].Applied Soft Computing,2020,94:1-10.
[25]BEHERA R K,JENA M,RATH S K,et al.Co-LSTM:Convolutional LSTM model for sentiment analysis in social big data[J].Information Processing & Management,2021,58(1):1-18.
[26]BASIRI M E,NEMATI S,ABDAR M,et al.ABCDM:An attention-based bidirectional CNN-RNN deep model for sentiment analysis[J].Future Generation Computer Systems,2021,115:279-294.
[27]GAN C Q,FENG Q D,ZHANG Z F.Scalable multi-channel dilated CNN-BiLSTM model with attention mechanism for Chinese textual sentiment analysis[J].Future Generation Compu-ter Systems,2021,118:297-309.
[28]LI W J,QI F,TANG M,et al.Bidirectional LSTM with self-attention mechanism and multi-channel features for sentiment classification[J].Neurocomputing,2020,387:63-77.
[29]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A Convolutional Neural Network for Modelling Sentences[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2014:655-665.
[30]SHIN B,LEE T,CHOI J D.Lexicon integrated CNN models with attention for sentiment analysis[J/OL]. https://arxiv.org/abs/1610.06272.
[31]TAI K S,SOCHER R,MANNING C D.Improved SemanticRepresentations From Tree-Structured Long Short-Term Me-mory Networks[C]//Proceedings of the 53nd Annual Meeting of the Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2015:1556-1566.
[32]CHEN C T,ZHOU R,REN J T.Gated recurrent neural network with sentimental relations for sentiment classification[J].Information Sciences,2019,502:268-278.
[33]ZHOU L,BIAN X Y.Improved text sentiment classicationmethod based on BiGRU-Attention[J]. Journal of Physics:Conference Series,2019,1345:032097.
[34]YANG L,LI Y,WANG J,et al.Sentiment analysis for e-commerce product reviews in Chinese based on sentiment lexicon and deep learning[J].IEEE Access,2020,8:23522-23530.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!