基于多级多尺度特征提取的CNN-BiLSTM模型的中文情感分析

doi:10.11896/jsjkx.220400069

摘要/Abstract

摘要： 情感分析作为自然语言处理(NLP)的一个研究子领域,在舆情监测方面起着非常重要的作用。在中文情感分析任务中,已有方法仅从单极、单尺度来考虑情感特征,无法充分挖掘和利用情感特征信息,模型性能不理想。针对这一问题,提出了一种多级多尺度特征提取的CNN-BiLSTM模型。该模型首先利用预训练好的中文词向量模型并结合嵌入层微调来获取词级特征;然后利用多尺度短语级特征表征模块和句子级特征表征模块来分别获取短语级和句子级特征,在多尺度短语级特征表征模块中,使用具有不同卷积核尺寸的卷积网络来获取不同尺度的短语级特征;最后使用多级特征融合方法将词级特征、不同尺度的短语级特征以及句子级特征进行融合形成多级联合特征,与单极、单尺度特征相比,多级联合特征具有更多的情感信息。在实验中,使用Accuracy,Precision,Recall,F1这4个评估指标对模型性能进行评估,并与包括支持向量机(SVM)在内的8种方法进行比较。实验结果表明,所提方法在4个评估指标中的得分均优于8种对比方法,证明了所提模型在多级和多尺度特征提取上的优势。

关键词: 自然语言处理, 中文情感分析, 多级多尺度特征, 卷积神经网络, 双向长短期记忆网络

Abstract: Sentiment analysis,as a sub-field of natural language processing(NLP),plays a very important role in public opinion monitoring.In the Chinese sentiment analysis task,the existing methods only consider sentiment features from single-level and single-scale,which cannot fully mine and utilize the sentiment feature information,and the performance of the model is not ideal.To solve this problem,a CNN-BiLSTM model with multi-level and multi-scale feature extraction is proposed.This model first uses a pre-trained Chinese word vector model combined with embedding layer fine-tuning to obtain word-level features.Then,phrase-level and sentence-level features are obtained by multi-scale phrase-level feature representation module and sentence-level feature representation module respectively.In the multi-scale phrase-level feature representation module,convolutional networks with different convolution kernel sizes are used to obtain phrase-level features of different scales.Finally,a multi-level feature fusion method is used to fuse word-level features,phrase-level features of different scales,and sentence-level features to form multi-level joint features.Compared with single-level and single-scale features,multi-level joint features have more sentiment information.In the experiment,four evaluation indicators(Accuracy,Precision,Recall,F1) are used to evaluate the performance of the model and compared with eight methods including support vector machines(SVM).Experimental results show that the proposed method outperforms the eight comparison methods in the four evaluation indicators,which proves the advantages of the proposed model in multi-level and multi-scale feature extraction.

Key words: Natural language processing, Chinese sentiment analysis, Multi-level and multi-scale feature, Convolutional neural network, Bidirectional long short-term memory network

中图分类号:

TP391

汪林, 蒙祖强, 杨丽娜. 基于多级多尺度特征提取的CNN-BiLSTM模型的中文情感分析[J]. 计算机科学, 2023, 50(5): 248-254. https://doi.org/10.11896/jsjkx.220400069

WANG Lin, MENG Zuqiang, YANG Lina. Chinese Sentiment Analysis Based on CNN-BiLSTM Model of Multi-level and Multi-scale Feature Extraction[J]. Computer Science, 2023, 50(5): 248-254. https://doi.org/10.11896/jsjkx.220400069

参考文献

[1]WANG L,NIU J W,YU S.Combining Textual Information and Sentiment Diffusion Patterns for Twitter Sentiment Analysis[J].IEEE Transactions on Knowledge and Data Engineering,2020,32(10):2026-2039.
[2]HASSONAH M R,AL-SAYYED R,RODAN A,et al.An effi-cient hybrid filter and evolutionary wrapper approach for sentiment analysis of various topics on Twitter[J].Knowledge-Based Systems,2020,192:1-19.
[3]KAGAN V,STEVENS A,SUBRAHMANIAN V S.UsingTwitter Sentiment to Forecast the 2013 Pakistani Election and the 2014 Indian Election[J].IEEE Intelligent Systems,2015,30(1):2-5.
[4]BACCIANELLA S,ESULI A,SEBASTIANI F.SENTIWORDNET 3.0:An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining[C]//Proceedings of the 7th Interna-tional Conference on Language Resources and Evaluation.Valletta:ELRA,2010:55-57.
[5]DENG S Y,SINHA A P,ZHAO H M.Adapting sentiment lexi-cons to domain-specific social media texts[J].Decision Support Systems,2017,94:65-76.
[6]ASGHAR M Z,KHAN A,AHMAD S,et al.Lexicon-enhanced sentiment analysis framework using rule-based classification scheme[J].Plos One,2017,12(2):1-22.
[7]HAN H Y,ZHANG J P,YANG J,et al.Generate domain-specific sentiment lexicon for review sentiment analysis[J].Multimedia Tools and Applications,2018,77(16):21265-21280.
[8]CAI Y,YANG K,HUANG D P,et al.A hybrid model for opi-nion mining based on domain sentiment dictionary[J].International Journal of Machine Learning and Cybernetics,2019,10(8):2131-2142.
[9]HAJEK P.Combining bag-of-words and sentiment features of annual reports to predict abnormal stock returns[J].Neural Computing & Applications,2018,29(7):343-358.
[10]DEY A,JENAMANI M,THAKKAR J J.Lexical TF-IDF:An n-gram Feature Space for Cross-Domain Classification of Sentiment Reviews[C]//Proceedings of the 7th International Confe-rence on Pattern Recognition and Machine Intelligence.Switzerland:Springer International Publishing,2017:380-386.
[11]MANEK A S,SHENOY P D,MOHAN M C,et al.Aspect term extraction for sentiment analysis in large movie reviews using Gini Index feature selection method and SVM classifier[J].World Wide Web-Internet and Web Information Systems,2017,20(2):135-154.
[12]ROUT J K,CHOO K K R,DASH A K,et al.A model for sentiment and emotion analysis of unstructured social media text[J].Electronic Commerce Research,2018,18(1):181-199.
[13]HUQ M R,ALI A,RAHMAN A.Sentiment Analysis on Twitter Data using KNN and SVM[J].International Journal of Advanced Computer Science and Applications,2017,8(6):19-25.
[14]LI Q,LI P F,MAO K Z,et al.Improving convolutional neural network for text classification by recursive data pruning[J].Neurocomputing,2020,414:143-152.
[15]JELODAR H,WANG Y L,ORJI R,et al.Deep Sentiment Classification and Topic Discovery on Novel Coronavirus or COVID-19 Online Discussions:NLP Using LSTM Recurrent Neural Network Approach[J].IEEE Journal of Biomedical and Health Informatics,2020,24(10):2733-2742.
[16]GAN C Q,WANG L,ZHANG Z F,et al.Sparse attention based separable dilated convolutional neural network for targeted sentiment analysis[J].Knowledge-Based Systems,2020,188:1-10.
[17]ZHAO P L,HOU L L,WU O.Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification[J].Knowledge-Based Systems,2020,193:1-10.
[18]ZHOU J,HUANG J X,HU Q V,et al.SK-GCN:Modeling Syntax and Knowledge via Graph Convolutional Network for aspect-level sentiment classification[J].Knowledge-Based Systems,2020,205:1-10.
[19]HASSAN J,SHOAIB U.Multi-class Review Rating Classification using Deep Recurrent Neural Network[J].Neural Proces-sing Letters,2020,51(1):1031-1048.
[20]AHMED M,CHEN Q,LI Z H.Constructing domain-dependent sentiment dictionary for sentiment analysis[J].Neural Computing & Applications,2020,32(18):14719-14732.
[21]WEI J Y,LIAO J,YANG Z F,et al.BiLSTM with Multi-Polarity Orthogonal Attention for Implicit Sentiment Analysis[J].Neurocomputing,2020,383:165-173.
[22]ZHANG Y,WANG J,ZHANG X.Conciseness is better:Recurrent attention LSTM model for document-level sentiment analysis[J].Neurocomputing,2021,462:101-112.
[23]WANG B,SHAN D,FAN A,et al.A Sentiment Classification Method of Web Social Media Based on Multidimensional and Multilevel Modeling[J].IEEE Transactions on Industrial Informatics,2021,18(2):1240-1249.
[24]LI W,ZHU Y,SHI Y,et al.User reviews:Sentiment analysisusing lexicon integrated two-channel CNN-LSTM family models[J].Applied Soft Computing,2020,94:1-10.
[25]BEHERA R K,JENA M,RATH S K,et al.Co-LSTM:Convolutional LSTM model for sentiment analysis in social big data[J].Information Processing & Management,2021,58(1):1-18.
[26]BASIRI M E,NEMATI S,ABDAR M,et al.ABCDM:An attention-based bidirectional CNN-RNN deep model for sentiment analysis[J].Future Generation Computer Systems,2021,115:279-294.
[27]GAN C Q,FENG Q D,ZHANG Z F.Scalable multi-channel dilated CNN-BiLSTM model with attention mechanism for Chinese textual sentiment analysis[J].Future Generation Compu-ter Systems,2021,118:297-309.
[28]LI W J,QI F,TANG M,et al.Bidirectional LSTM with self-attention mechanism and multi-channel features for sentiment classification[J].Neurocomputing,2020,387:63-77.
[29]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A Convolutional Neural Network for Modelling Sentences[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2014:655-665.
[30]SHIN B,LEE T,CHOI J D.Lexicon integrated CNN models with attention for sentiment analysis[J/OL]. https://arxiv.org/abs/1610.06272.
[31]TAI K S,SOCHER R,MANNING C D.Improved SemanticRepresentations From Tree-Structured Long Short-Term Me-mory Networks[C]//Proceedings of the 53nd Annual Meeting of the Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2015:1556-1566.
[32]CHEN C T,ZHOU R,REN J T.Gated recurrent neural network with sentimental relations for sentiment classification[J].Information Sciences,2019,502:268-278.
[33]ZHOU L,BIAN X Y.Improved text sentiment classicationmethod based on BiGRU-Attention[J]. Journal of Physics:Conference Series,2019,1345:032097.
[34]YANG L,LI Y,WANG J,et al.Sentiment analysis for e-commerce product reviews in Chinese based on sentiment lexicon and deep learning[J].IEEE Access,2020,8:23522-23530.

相关文章 15

[1]	张雪, 赵晖. 基于多事件语义增强的情感分析 Sentiment Analysis Based on Multi-event Semantic Enhancement 计算机科学, 2023, 50(5): 238-247. https://doi.org/10.11896/jsjkx.220400256
[2]	叶瀚, 李欣, 孙海春. 结合门控机制的卷积网络实体缺失检测方法 Convolutional Network Entity Missing Detection Method Combined with Gated Mechanism 计算机科学, 2023, 50(5): 262-269. https://doi.org/10.11896/jsjkx.220400126
[3]	常利伟, 刘秀娟, 钱宇华, 耿海军, 赖裕平. 基于卷积神经网络多源融合的网络安全态势感知模型 Multi-source Fusion Network Security Situation Awareness Model Based on Convolutional Neural Network 计算机科学, 2023, 50(5): 382-389. https://doi.org/10.11896/jsjkx.220400134
[4]	邵云飞, 宋友, 王宝会. 基于社交网络图节点度的神经网络个性化传播算法研究 Study on Degree of Node Based Personalized Propagation of Neural Predictions forSocial Networks 计算机科学, 2023, 50(4): 16-21. https://doi.org/10.11896/jsjkx.220300274
[5]	王振彪, 覃亚丽, 王荣芳, 郑欢. 基于残差特征聚合的图像压缩感知注意力神经网络 Image Compressed Sensing Attention Neural Network Based on Residual Feature Aggregation 计算机科学, 2023, 50(4): 117-124. https://doi.org/10.11896/jsjkx.211200215
[6]	甄田歌, 宋明阳, 景丽萍. 融合多粒度抽取式特征的关键词生成 Incorporating Multi-granularity Extractive Features for Keyphrase Generation 计算机科学, 2023, 50(4): 181-187. https://doi.org/10.11896/jsjkx.220700164
[7]	曹晨阳, 杨晓东, 段鹏松. WiDoor:一种近距离非接触式身份识别方法 WiDoor:Close-range Contactless Human Identification Approach 计算机科学, 2023, 50(4): 388-396. https://doi.org/10.11896/jsjkx.220300278
[8]	李帅, 徐彬, 韩祎珂, 廖同鑫. SS-GCN:情感增强和句法增强的方面级情感分析模型 SS-GCN:Aspect-based Sentiment Analysis Model with Affective Enhancement and Syntactic Enhancement 计算机科学, 2023, 50(3): 3-11. https://doi.org/10.11896/jsjkx.220700238
[9]	王晓飞, 樊学强, 李章维. 基于迁移学习和多视图特征融合提高RNA碱基相互作用预测 Improving RNA Base Interactions Prediction Based on Transfer Learning and Multi-view Feature Fusion 计算机科学, 2023, 50(3): 164-172. https://doi.org/10.11896/jsjkx.211200186
[10]	梅鹏程, 杨吉斌, 张强, 黄翔. 一种基于三维卷积的声学事件联合估计方法 Sound Event Joint Estimation Method Based on Three-dimension Convolution 计算机科学, 2023, 50(3): 191-198. https://doi.org/10.11896/jsjkx.220500259
[11]	曹金娟, 钱忠, 李培峰. 基于联合模型的端到端事件可信度识别 End-to-End Event Factuality Identification with Joint Model 计算机科学, 2023, 50(2): 292-299. https://doi.org/10.11896/jsjkx.211200108
[12]	孙凯丽, 罗旭东, 罗有容. 预训练语言模型的应用综述 Survey of Applications of Pretrained Language Models 计算机科学, 2023, 50(1): 176-184. https://doi.org/10.11896/jsjkx.220800223
[13]	郑诚, 梅亮, 赵伊研, 张苏航. 基于双向注意力机制和门控图卷积网络的文本分类方法 Text Classification Method Based on Bidirectional Attention and Gated Graph Convolutional Networks 计算机科学, 2023, 50(1): 221-228. https://doi.org/10.11896/jsjkx.211100095
[14]	周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[15]	陈泳全, 姜瑛. 基于卷积神经网络的APP用户行为分析方法 Analysis Method of APP User Behavior Based on Convolutional Neural Network 计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed