计算机科学 ›› 2020, Vol. 47 ›› Issue (11A): 28-34.doi: 10.11896/jsjkx.191100114

• 人工智能 • 上一篇    下一篇

基于上下文的情感词向量混合模型

霍丹1, 张生杰2, 万路军1   

  1. 1 空军工程大学 西安 710048
    2 西安理工大学计算机科学与工程学院 西安 710048
  • 出版日期:2020-11-15 发布日期:2020-11-17
  • 通讯作者: 张生杰(754272317@qq.com)
  • 作者简介:wan-95@163.com
  • 基金资助:
    国家自然科学基金(61703452);陕西省自然科学研究发展计划(2016JQ6062)

Context-based Emotional Word Vector Hybrid Model

HUO Dan1, ZHANG Sheng-jie2, WAN Lu-jun1   

  1. 1 Air Force Engineering University,Xi'an 710048,China
    2 School of Computer Science and Engineering,Xi'an University of Technology,Xi'an 710048,China
  • Online:2020-11-15 Published:2020-11-17
  • About author:HUO Dan,born in 1990,M.S.,lecturer.Her main research interests include natural language processing and so on.
    ZHANG Sheng-jie,born in 1986,master,engineer.His main research inte-rests include big data processing and content security.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61703452) and Research Plan of the National Natural Science and Development of Shaanxi Province,China (2016JQ6062).

摘要: 针对现有大多数基于词向量的学习方法只能对词语的语法语境建模,而忽略了词语的情感信息的问题,文中提出了基于上下文的情感词向量训练模型,使用了比较简单的方法来构建情感词向量的学习框架。该模型是能够获取情感的扩展混合模型在句子极性的情感信息和基于上下文级别词向量的融合方法,有效解决了具有相似上下文但相反情感极性的词被映射到相邻的词向量的问题。为验证学习到的情感词向量模型能准确包含情感和上下文词语的语义信息,分别在不同的语言和不同领域的数据集下训练情感词向量,并在词语级别进行了定量实验。结果表明,所提的情感词向量学习模型在情感词向量获取实验中,与传统的词向量学习模型相比,分类效果提升了14个百分点;在词语级别的情感分类实验中,与传统的词袋模型相比,准确性提升了10个百分点,从而也对产品提供商在大量的用户评价中得到有用的信息起到了指导性的作用。

关键词: 词向量, 情感分类, 神经网络, 语义信息, 自然语言处理

Abstract: Most of the existing learning methods based on word vectors can only model the syntactic context of words,but ignore the emotional information of words.This paper proposes a context-based training model of emotional word vectors,and uses a rela-tively simple method to construct a learning framework of emotional word vectors.A fusion method is proposed to obtain the emotion information of the extended mixed model in the sentence polarity and the context-based word vectors.So as to solve the problem that words with similar contexts but opposite emotional polarity are mapped to adjacent word vectors.the adjacent words in the emotion vector space are semantically similar and have the same emotion polarity.In order to verify that the learned emotion word vector model can accurately contain the semantic information of emotion and context words,the emotion word vector is trained in different languages and data sets of different fields,and quantitative experiments are conducted at the word level.The results show that the classification effect of the proposed model is 14 percent higher than that of the traditional model.In the experiment of emotion classification at the word level,the accuracy is improved by 10 percentage points compared with the traditional word bag model.It also plays a guiding role for product providers to get useful information in a large number of user reviews.

Key words: Natural language processing, Neural networks, Semantic information, Sentiment classification, Word embeddings

中图分类号: 

  • TP389.1
[1] LAI S W.Word and Document Embeddings based on NeuralNetwork Approaches[D].Beijing:Institute of Automation Chinese Academy of Sciences,2016.
[2] THOMAS K L,PETER W F,DARRELL L.An introduction to latent semantic analysis[J].Discourse processes,1988,25(2/3):259-284.
[3] MICHAEL N J,DOUGLAS J K M.Representing word meaning and order information in a composite holographic lexicon[J].Psychological review,2007:114(1):301-311.
[4] SOCHER R,PERELYGIN A,WU J.Recursive Deep Models for Semantic Composition- ality Over a Sentiment Treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.2013:1631-1642.
[5] YESSENALINA A,CARDIE C.Compositional matrix-spacemodels for sentiment analysis[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Proces-sing.2011:172-182.
[6] COLLOBERT R,WESTON J,BOTTOU L.Natural Language Processing (Almost) from Scratch[J].Journal of Machine Learning Research,2011:2493-2537.
[7] ZHENG X,CHEN H,XU T.Deep learning for Chinese word segmentation and POS tagging[C]//Proceeings of the 2013 Conference on Empirical Methods in Natural Language Proces-sing.2013:647-657.
[8] TANG D,WEI F,YANG N,et al.Learning entiment-specificword embedding for twitter sentiment classification[C]//Proceedings of the 2014/ 52th Annu.Meeting Assoc.Computer.Linguistics,2014:1555-1565.
[9] BENGIO Y,DUCHARME R,VINCENT P.A neural probabilistic language model[J].Journal of Machine Learning Research,2003:1137-1155.
[10] LUO Y,LI L,TAN S,et al.Sentiment analysis on Chinese Micro-blog corpus[J].Journal of Shandong University (Natural Science),2014,49(11):1-7.
[11] CHEN X D.Research on Sentiment Dictionary based Emotional Tendency Analysis of Chinese MicroBlog[D].Wuhan:Huazhong University of Science & Technology,2012.
[12] TURIAN J,RATINOV L,BENGIO Y.Word representations:a simple and general method for semi-supervised learning[C]//ACL 2010,Proceedings of the,Meeting of the Association for Computational Linguistics,Uppsala,Sweden.DBLP,2010:384-394.
[13] PENNINGTON J,SOCHER R,MANNING C.Glove:GlobalVectors for Word Representation[C]//Conference on Empirical Methods in Natural Language Processing.2014:1532-1543.
[14] MIKOLOV T,CHEN K,CORRADO G,et al.Efficient Estimation of Word Representations in Vector Space[J].Computer Science,2013:566-569.
[15] IRSOY O,CARDIE C.Opinion Mining with Deep RecurrentNeural Networks[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.2014:720-728.
[16] PILEHVAR M T,COLLIER N.Improved Semantic Representation for Domain-Specific Entities[C]//Proceedings of the 15th Workshop on Biomedical Natural Language Processing.2016:13-16.
[17] PANG B,LEE L,VAITHYANATHAN S.Thumbs up Sentiment Classification using Machine Learning Techniques[C]//Proceeings of the 2002 Conference on Clinical Orthopaedics and Related Research.2002:79-86.
[18] ANDREW L M,RAYMOND E D,PETER T P,et al.Learning word vectors for sentiment analysis[C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL).2011:142-150.
[19] JEFFREY P,RICHARD S,CHRISTOPHER D M.GloVe:Global Vectors for Word Representation[C]//Proceedings of the 2014 Empiricial Methods in Natural Language Processing.Washington,DC:IEEE Computer Society,2014:1201-1220.
[20] BENGIO Y,COURVILLE A,VINCENT P.RepresentationLearning:A Review and New Perspectives[J].IEEE Trans on Pattern Analysis and Machine Intelligence,2013,35(8):1798-1828.
[21] MIKOLOV T,SUTSKEVER I,CHEN T,et al.Distributed represen- tations of words and phrases and their compositionality[C]//Proceedings of the 2013/15th International Conference on Neural Information Processing Systems.2013:3111-3119.
[22] BRIDLE J.Probabilistic interpretation of feed forward classification network outputs,with relationships to statistical pattern recognition in Neurocomputing[C]//Proceedings of the 1990 Neurocomputing:Algorithms,Architectures and Applications.Berlin:Springer,1990:227-236.
[23] COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing (almost) from scratch[J].Journal of Machine Learning Research,2011,12:2493-2537.
[24] MNIH A,KAVUKCUOGLU K.Learning word embeddings efficiently with noise- contrastive estimation[J].Advances in Neural Information Processing Systems,2013:2265-2273.
[25] BARONI M,DINU G,KRUSZEWSKI G.Don't count,predict! A systematic comparison of Context-counting vs.Context-predicting semantic vectors[C]//Proceedings of the 2014/ 52th Annu.Meeting Assoc.Computer Linguistics.2014:238-247.
[26] DUCHI J,HAZAN E,SINGER Y.Adaptive subgradient metho-ds for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011,12(2):2121-2159.
[27] SOCHER R,PERELYGIN A,WU J,et al.Recursive deep mo-dels for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference Empirical Methods Natural Lang.2013:1631-1642.
[28] ZHANG W,SHI W X,LIU D N,et al.Improved Approach of Word Vector Learning Via Sentiment Information[J].Application Research of Computers,2016,34(8):120-130.
[29] RUMELHART D,HINTON G,WILLIAMS R.Learning representations by back-propagating errors[J].Cognitive Modeling,1988,5:1003-1120.
[30] LABUTOV L,LIPSON H.Reembedding words[C]//Procee-dings of the 2013 Meeting of Association for Computational Linguistics (ACL).2013:489-493.
[31] COLLOBERT R,WESTON J.A unified architecture for natural language processing:Deep neural networks with multitask lear-ning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2008:160-167.
[32] TANG D,QIN B,LIU T,et al.User modeling with neural network for review rating prediction[C]//Proceedings of the 2015/24th International Conference Analysis and Machine Intelligence.2015:1340-1346.
[1] 周芳泉, 成卫青.
基于全局增强图神经网络的序列推荐
Sequence Recommendation Based on Global Enhanced Graph Neural Network
计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[2] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[3] 宁晗阳, 马苗, 杨波, 刘士昌.
密码学智能化研究进展与分析
Research Progress and Analysis on Intelligent Cryptology
计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053
[4] 王润安, 邹兆年.
基于物理操作级模型的查询执行时间预测方法
Query Performance Prediction Based on Physical Operation-level Models
计算机科学, 2022, 49(8): 49-55. https://doi.org/10.11896/jsjkx.210700074
[5] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[6] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[7] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[8] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[9] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[10] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[11] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[12] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[13] 姜胜腾, 张亦弛, 罗鹏, 刘月玲, 曹阔, 赵海涛, 魏急波.
语义通信系统的性能度量指标分析
Analysis of Performance Metrics of Semantic Communication Systems
计算机科学, 2022, 49(7): 236-241. https://doi.org/10.11896/jsjkx.211200071
[14] 彭双, 伍江江, 陈浩, 杜春, 李军.
基于注意力神经网络的对地观测卫星星上自主任务规划方法
Satellite Onboard Observation Task Planning Based on Attention Neural Network
计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093
[15] 费星瑞, 谢逸.
基于HMM-NN的用户点击流识别
Click Streams Recognition for Web Users Based on HMM-NN
计算机科学, 2022, 49(7): 340-349. https://doi.org/10.11896/jsjkx.210600127
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!