基于词嵌入辅助机制的情感分析

doi:10.11896/jsjkx.180901687

计算机科学 ›› 2019, Vol. 46 ›› Issue (10): 258-264.doi: 10.11896/jsjkx.180901687

基于词嵌入辅助机制的情感分析

韩旭丽¹, 曾碧卿², 曾锋¹, 张敏¹, 商齐¹

(华南师范大学计算机学院广州 510631)¹
(华南师范大学软件学院广东佛山528225)²

收稿日期:2018-09-09 修回日期:2018-12-22 出版日期:2019-10-15 发布日期:2019-10-21
通讯作者: 曾碧卿(1969-),男,博士,教授,CCF高级会员,主要研究方向为自然语言处理,E-mail:zengbiqing0528@163.com。
作者简介:韩旭丽(1992-),女,硕士生,主要研究方向为自然语言处理、情感分析;曾锋(1992-),男,硕士生,主要研究方向为自然语言处理;张敏(1991-),女,硕士生,主要研究方向为自然语言处理;商齐(1994-),男,主要研究方向为自然语言处理。
基金资助:
本文受国家自然科学基金项目(61772211,61503143)资助。

Sentiment Analysis Based on Word Embedding Auxiliary Mechanism

HAN Xu-li¹, ZENG Bi-qing², ZENG Feng¹, ZHANG Min¹, SHANG Qi¹

(School of Computer Science,South China Normal University,Guangzhou 510631,China)¹
(School of Software,South China Normal University,Foshan,Guangdong 528225,China)²

Received:2018-09-09 Revised:2018-12-22 Online:2019-10-15 Published:2019-10-21

摘要/Abstract

摘要： 文本情感分析是自然语言处理研究领域中一个重要的研究方向,如何分析出长文本的情感极性是一个研究难点。目前,大部分研究工作倾向于将词嵌入应用在神经网络模型中进行情感分析,虽然这种方法的词特征表示能力较好,但是对于长文本来说有待优化,过长的文本会给模型带来沉重的负担,使模型在训练过程中耗费更多的时间和计算资源。针对此问题,文中提出了一种基于词嵌入辅助机制的注意力神经网络模型(Word Embedding Auxiliary Mechanism Based Attentional Neural Network Model,WEAN),并将其应用于长文本的情感分析任务。该模型使用词嵌入辅助机制解决了长文本在神经网络模型中的训练负担问题,利用双向循环神经网络获取序列中的上下文信息,同时应用注意力机制来捕获序列中不同重要程度的信息,提高了情感分类的性能。在IMDB,Yelp 2013和Yelp 2014数据集上的实验结果表明,与NSC+LA模型相比,所提模型的情感分析准确率分别提高了1.1%,2.0%和2.6%。

关键词: 词嵌入, 情感分析, 神经网络, 注意力机制, 自然语言处理

Abstract: Text sentiment analysis is an important research direction in natural language processing,and how to analyze the sentiment polarity of long text is a research hotspot.At present,majority of the researches tend to apply word embedding in neural network models for sentiment analysis.Although this method has a good representation ability of word features,it has some weaknesses for long text.Extremely long text will bring heavy burden to the model and make it consume more time and resources in training process.In light of this,this paper proposed an attention neural network model based on word embedding auxiliary mechanism,namely WEAN,and it is applied to sentiment analysis for long text.The model deals with some training burdens of long text in neural network model by using word embedding auxi-liary mechanism and utilizes bidirectional recurrent neural network and mechanism to obtain context information.At the same time,it captures information with different importance degree in sequences,thus improving the sentiment classification performance.Experiment was conducted on IMDB,Yelp 2013 and Yelp 2014 datasets.The results show that the accuracy of the sentiment analysis of the proposed model increases by 1.1%,2.0% and 2.6% respectively on three datasets compared with NSC+LA model.

Key words: Attention mechanism, Natural language processing, Neural network, Sentiment analysis, Word embedding

中图分类号:

TP391

韩旭丽, 曾碧卿, 曾锋, 张敏, 商齐. 基于词嵌入辅助机制的情感分析[J]. 计算机科学, 2019, 46(10): 258-264. https://doi.org/10.11896/jsjkx.180901687

HAN Xu-li, ZENG Bi-qing, ZENG Feng, ZHANG Min, SHANG Qi. Sentiment Analysis Based on Word Embedding Auxiliary Mechanism[J]. Computer Science, 2019, 46(10): 258-264. https://doi.org/10.11896/jsjkx.180901687

参考文献

[1]WANG S,MANNING C D.Baselines and bigrams:simple,good sentiment and topic classification[C]//Meeting of the Association for Computational Linguistics:Short Papers.Jeju Island,Korea:ACL 2012:90-94.
[2]BENGIO Y.Learning Deep Architectures for AI [J].Founda-tions & Trends^® in Machine Learning,2009,2(1):1-127.
[3]SOCHER R,LIN C,MANNING C D,et al.Parsing Natural Scenes and Natural Language with Recursive Neural Networks[C]//International Conference on Machine Learning.Bellevue,Washington:ICML,2011:129-136.
[4]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013.
[5]HOCHREITER S,SCHMIDHUBER J.Long short-term memory [J].Neural Computation,1997,9(8):1735-1780.
[6]KIM Y.Convolutional Neural Networks for Sentence Classification[J].Empirical Methods in Natural Language Processing,2014,1:1746-1751.
[7]ZHANG Y,GAN Z,FAN K,et al.Adversarial Feature Matching for Text Generation[C]//International Conference on Machine Learning.Sydney:ICAL,2017:4006-4015.
[8]GRAVES A,JAITLY N,MOHAMED A,et al.Hybrid speech recognition with Deep Bidirectional LSTM[C]//IEEE Automa-tic Speech Recognition and Understanding Workshop.Olomouc,Czech Republic:IEEE,2013:273-278.
[9]YANG Z,YANG D,DYER C,et al.Hierarchical Attention Networks for Document Classification[C]//Northamerican Chapter of the Association for Computational Linguistics.San Diego,California:ACL,2016:1480-1489.
[10]PANG B,LEE L,VAITHYANATHAN S,et al.Thumbs up? Sentiment Classification using Machine Learning Techniques[J].Empirical Methods in Natural Language Processing,2002,1(1):79-86.
[11]TURNEY P D.Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews[J].Meeting of the Association for Computational Linguistics,2002,10:417-424.
[12]CONNEAU A,SCHWENK H,BARRAULT L,et al.Very deep convolutional networks for text classification[J].Conference of the European Chapter of the Association for Computational Linguistics,2017,1:1107-1116.
[13]SHEN D,MIN M R,LI Y,et al.Adaptive convolutional filter generation for natural language understanding[J].arXiv:2017,1709.08294.
[14]SHI B,FU Z,BING L,et al.Learning Domain-Sensitive and Sentiment-Aware Word Embeddings[J].Meeting of the Association for Computational Linguistics,2018,1(1):2494-2504.
[15]ADI Y,KERMANY E,BELINKOV Y,et al.Fine-grained Ana-lysis of Sentence Embeddings Using Auxiliary Prediction Tasks[J/OL].International Conference on Learning Representations,2017.https://openreview.net/group?id=ICLR.cc/2017/conference.
[16]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P,et al.A Convolutional Neural Network for Modelling Sentences[J].Meeting of the Association for Computational Linguistics,2014,1:655-665.
[17]JOHNSON R,ZHANG T.Effective Use of Word Order for Text Categorization with Convolutional Neural Networks[OL].http://xxx.tau.ac.il/pdf/1412.1058.pdf.
[18]ZHANG X,ZHAO J,LECUN Y.Character-level c-onvolutional networks for text classification [C]//Advances in Neural Information Processing Systems.Montreal,Canada:NPIS,2015:649-657.
[19]LI J,LUONG T,JURAFSKY D,et al.When Are Tree Struc-tures Necessary for Deep Learning of Representations[J].Empiricalmethods in Natural Language Processing,2015,1(1):2304-2314.
[20]LE Q V,MIKOLOV T.Distributed Representations of Sen-tences and Documents[OL].http://people.ee.duke.edu/~lcarin/ChunyuanLi4.17.2015.pdf.
[21]TANG D,QIN B,LIU T.Document Modeling with Gated Recurrent Neural Network for Sentiment Classification[C]//Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal:ACL,2015:1422-1432.
[22]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is All you Need[C]//Neural Information Processing Systems.Long Beach:NIPS,2017:5998-6008.
[23]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTM for Aspect-level Sentiment Classification[C]//Conference on Empirical Methods in Natural Language Processing.Austin,Texas:ACL,2017:606-615.
[24]MNIH V,HEESS N,GRAVES A.Recurrent models of visual attention[C]//Advances in Neural Information Processing Systems.Montreal,Canada:NPIS,2014:2204-2212.
[25]BAHDANAU D,CHO K,BENGIO Y,et al.Neural Machine Translation by Jointly Learning to Align and Translate[OL].https://arxiv-vanity.com/papers/1409.0473.
[26]LUONG T,PHAM H,MANNING C D,et al.Effective Approaches to Attention-based Neural Machine Translation[J].Empi-rical Methods in Natural Language Processing,2015,1:1412-1421.
[27]FAN R E,CHANG K W,HSIEH C J,et al.LIBLINEAR:A Library for Large Linear Classification [J].Journal of Machine Learning Research,2008,9(9):1871-1874.
[28]KIRITCHENKO S,ZHU X,MOHAMMAD S M,et al.Senti-ment analysis of short informal texts[J].Journal of Artificial Intelligence Research,2014,50(1):723-762.
[29]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed Representations of Words and Phrases and their compositionality[J].Advances in Neural Information Processing Systems,2013,26:3111-3119.
[30]TANG D,WEI F,YANG N,et al.Learning sentiment-specific word embedding for twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.Baltimore,Maryland,USA:ACL,2014,1:1555-1565.
[31]SOCHER R,PERELYGIN A,WU J,et al.Recursive deep mo-dels for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.Seattle,Washington,USA:ACL.2013:1631-1642.
[32]LE Q,MIKOLOV T.Distributed representations of sentences and documents[C]//International Conference on Machine Learning.Beijing,China:ICAL,2014:1188-1196.

相关文章 15

[1]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2]	宁晗阳, 马苗, 杨波, 刘士昌. 密码学智能化研究进展与分析 Research Progress and Analysis on Intelligent Cryptology 计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053
[3]	周芳泉, 成卫青. 基于全局增强图神经网络的序列推荐 Sequence Recommendation Based on Global Enhanced Graph Neural Network 计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[4]	戴禹, 许林峰. 基于文本行匹配的跨图文本阅读方法 Cross-image Text Reading Method Based on Text Line Matching 计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032
[5]	周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[6]	熊丽琴, 曹雷, 赖俊, 陈希亮. 基于值分解的多智能体深度强化学习综述 Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization 计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[7]	李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[8]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[9]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[10]	汪鸣, 彭舰, 黄飞虎. 基于多时间尺度时空图网络的交通流量预测模型 Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction 计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188
[11]	王润安, 邹兆年. 基于物理操作级模型的查询执行时间预测方法 Query Performance Prediction Based on Physical Operation-level Models 计算机科学, 2022, 49(8): 49-55. https://doi.org/10.11896/jsjkx.210700074
[12]	陈泳全, 姜瑛. 基于卷积神经网络的APP用户行为分析方法 Analysis Method of APP User Behavior Based on Convolutional Neural Network 计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[13]	朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[14]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[15]	檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于词嵌入辅助机制的情感分析

Sentiment Analysis Based on Word Embedding Auxiliary Mechanism

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0