Computer Science

Computer Science ›› 2020, Vol. 47 ›› Issue (11A): 28-34.doi: 10.11896/jsjkx.191100114

• Artificial Intelligence • Previous Articles     Next Articles

Context-based Emotional Word Vector Hybrid Model

HUO Dan1, ZHANG Sheng-jie2, WAN Lu-jun1   

  1. 1 Air Force Engineering University,Xi'an 710048,China
    2 School of Computer Science and Engineering,Xi'an University of Technology,Xi'an 710048,China
  • Online:2020-11-15 Published:2020-11-17
  • About author:HUO Dan,born in 1990,M.S.,lecturer.Her main research interests include natural language processing and so on.
    ZHANG Sheng-jie,born in 1986,master,engineer.His main research inte-rests include big data processing and content security.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61703452) and Research Plan of the National Natural Science and Development of Shaanxi Province,China (2016JQ6062).

PDF (PC)

1018

Abstract: Most of the existing learning methods based on word vectors can only model the syntactic context of words,but ignore the emotional information of words.This paper proposes a context-based training model of emotional word vectors,and uses a rela-tively simple method to construct a learning framework of emotional word vectors.A fusion method is proposed to obtain the emotion information of the extended mixed model in the sentence polarity and the context-based word vectors.So as to solve the problem that words with similar contexts but opposite emotional polarity are mapped to adjacent word vectors.the adjacent words in the emotion vector space are semantically similar and have the same emotion polarity.In order to verify that the learned emotion word vector model can accurately contain the semantic information of emotion and context words,the emotion word vector is trained in different languages and data sets of different fields,and quantitative experiments are conducted at the word level.The results show that the classification effect of the proposed model is 14 percent higher than that of the traditional model.In the experiment of emotion classification at the word level,the accuracy is improved by 10 percentage points compared with the traditional word bag model.It also plays a guiding role for product providers to get useful information in a large number of user reviews.

Key words: Natural language processing, Neural networks, Semantic information, Sentiment classification, Word embeddings

CLC Number: 

  • TP389.1
[1] LAI S W.Word and Document Embeddings based on NeuralNetwork Approaches[D].Beijing:Institute of Automation Chinese Academy of Sciences,2016.
[2] THOMAS K L,PETER W F,DARRELL L.An introduction to latent semantic analysis[J].Discourse processes,1988,25(2/3):259-284.
[3] MICHAEL N J,DOUGLAS J K M.Representing word meaning and order information in a composite holographic lexicon[J].Psychological review,2007:114(1):301-311.
[4] SOCHER R,PERELYGIN A,WU J.Recursive Deep Models for Semantic Composition- ality Over a Sentiment Treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.2013:1631-1642.
[5] YESSENALINA A,CARDIE C.Compositional matrix-spacemodels for sentiment analysis[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Proces-sing.2011:172-182.
[6] COLLOBERT R,WESTON J,BOTTOU L.Natural Language Processing (Almost) from Scratch[J].Journal of Machine Learning Research,2011:2493-2537.
[7] ZHENG X,CHEN H,XU T.Deep learning for Chinese word segmentation and POS tagging[C]//Proceeings of the 2013 Conference on Empirical Methods in Natural Language Proces-sing.2013:647-657.
[8] TANG D,WEI F,YANG N,et al.Learning entiment-specificword embedding for twitter sentiment classification[C]//Proceedings of the 2014/ 52th Annu.Meeting Assoc.Computer.Linguistics,2014:1555-1565.
[9] BENGIO Y,DUCHARME R,VINCENT P.A neural probabilistic language model[J].Journal of Machine Learning Research,2003:1137-1155.
[10] LUO Y,LI L,TAN S,et al.Sentiment analysis on Chinese Micro-blog corpus[J].Journal of Shandong University (Natural Science),2014,49(11):1-7.
[11] CHEN X D.Research on Sentiment Dictionary based Emotional Tendency Analysis of Chinese MicroBlog[D].Wuhan:Huazhong University of Science & Technology,2012.
[12] TURIAN J,RATINOV L,BENGIO Y.Word representations:a simple and general method for semi-supervised learning[C]//ACL 2010,Proceedings of the,Meeting of the Association for Computational Linguistics,Uppsala,Sweden.DBLP,2010:384-394.
[13] PENNINGTON J,SOCHER R,MANNING C.Glove:GlobalVectors for Word Representation[C]//Conference on Empirical Methods in Natural Language Processing.2014:1532-1543.
[14] MIKOLOV T,CHEN K,CORRADO G,et al.Efficient Estimation of Word Representations in Vector Space[J].Computer Science,2013:566-569.
[15] IRSOY O,CARDIE C.Opinion Mining with Deep RecurrentNeural Networks[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.2014:720-728.
[16] PILEHVAR M T,COLLIER N.Improved Semantic Representation for Domain-Specific Entities[C]//Proceedings of the 15th Workshop on Biomedical Natural Language Processing.2016:13-16.
[17] PANG B,LEE L,VAITHYANATHAN S.Thumbs up Sentiment Classification using Machine Learning Techniques[C]//Proceeings of the 2002 Conference on Clinical Orthopaedics and Related Research.2002:79-86.
[18] ANDREW L M,RAYMOND E D,PETER T P,et al.Learning word vectors for sentiment analysis[C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL).2011:142-150.
[19] JEFFREY P,RICHARD S,CHRISTOPHER D M.GloVe:Global Vectors for Word Representation[C]//Proceedings of the 2014 Empiricial Methods in Natural Language Processing.Washington,DC:IEEE Computer Society,2014:1201-1220.
[20] BENGIO Y,COURVILLE A,VINCENT P.RepresentationLearning:A Review and New Perspectives[J].IEEE Trans on Pattern Analysis and Machine Intelligence,2013,35(8):1798-1828.
[21] MIKOLOV T,SUTSKEVER I,CHEN T,et al.Distributed represen- tations of words and phrases and their compositionality[C]//Proceedings of the 2013/15th International Conference on Neural Information Processing Systems.2013:3111-3119.
[22] BRIDLE J.Probabilistic interpretation of feed forward classification network outputs,with relationships to statistical pattern recognition in Neurocomputing[C]//Proceedings of the 1990 Neurocomputing:Algorithms,Architectures and Applications.Berlin:Springer,1990:227-236.
[23] COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing (almost) from scratch[J].Journal of Machine Learning Research,2011,12:2493-2537.
[24] MNIH A,KAVUKCUOGLU K.Learning word embeddings efficiently with noise- contrastive estimation[J].Advances in Neural Information Processing Systems,2013:2265-2273.
[25] BARONI M,DINU G,KRUSZEWSKI G.Don't count,predict! A systematic comparison of Context-counting vs.Context-predicting semantic vectors[C]//Proceedings of the 2014/ 52th Annu.Meeting Assoc.Computer Linguistics.2014:238-247.
[26] DUCHI J,HAZAN E,SINGER Y.Adaptive subgradient metho-ds for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011,12(2):2121-2159.
[27] SOCHER R,PERELYGIN A,WU J,et al.Recursive deep mo-dels for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference Empirical Methods Natural Lang.2013:1631-1642.
[28] ZHANG W,SHI W X,LIU D N,et al.Improved Approach of Word Vector Learning Via Sentiment Information[J].Application Research of Computers,2016,34(8):120-130.
[29] RUMELHART D,HINTON G,WILLIAMS R.Learning representations by back-propagating errors[J].Cognitive Modeling,1988,5:1003-1120.
[30] LABUTOV L,LIPSON H.Reembedding words[C]//Procee-dings of the 2013 Meeting of Association for Computational Linguistics (ACL).2013:489-493.
[31] COLLOBERT R,WESTON J.A unified architecture for natural language processing:Deep neural networks with multitask lear-ning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2008:160-167.
[32] TANG D,QIN B,LIU T,et al.User modeling with neural network for review rating prediction[C]//Proceedings of the 2015/24th International Conference Analysis and Machine Intelligence.2015:1340-1346.
[1] NING Han-yang, MA Miao, YANG Bo, LIU Shi-chang. Research Progress and Analysis on Intelligent Cryptology [J]. Computer Science, 2022, 49(9): 288-296.
[2] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[3] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[4] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[5] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[6] WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
[7] SUN Jie-qi, LI Ya-feng, ZHANG Wen-bo, LIU Peng-hui. Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation [J]. Computer Science, 2022, 49(6A): 434-440.
[8] LI Xiao-wei, SHU Hui, GUANG Yan, ZHAI Yi, YANG Zi-ji. Survey of the Application of Natural Language Processing for Resume Analysis [J]. Computer Science, 2022, 49(6A): 66-73.
[9] LIN Xi, CHEN Zi-zhuo, WANG Zhong-qing. Aspect-level Sentiment Classification Based on Imbalanced Data and Ensemble Learning [J]. Computer Science, 2022, 49(6A): 144-149.
[10] GUO Liang, YANG Xing-yao, YU Jiong, HAN Chen, HUANG Zhong-hao. Hybrid Recommender System Based on Attention Mechanisms and Gating Network [J]. Computer Science, 2022, 49(6): 158-164.
[11] ZHANG Hong-min, LI Ping-ping, FANG Xiao-bing, LIU Hong. Human Abnormal Behavior Detection Method Based on Improved YOLOv3 Network Model [J]. Computer Science, 2022, 49(4): 233-238.
[12] LI Yong, WU Jing-peng, ZHANG Zhong-ying, ZHANG Qiang. Link Prediction for Node Featureless Networks Based on Faster Attention Mechanism [J]. Computer Science, 2022, 49(4): 43-48.
[13] PAN Zhi-hao, ZENG Bi, LIAO Wen-xiong, WEI Peng-fei, WEN Song. Interactive Attention Graph Convolutional Networks for Aspect-based Sentiment Classification [J]. Computer Science, 2022, 49(3): 294-300.
[14] SHAO Hai-lin, JI Yi, LIU Chun-ping, XU Yun-long. Scene Text Detection Algorithm Based on Enhanced Feature Pyramid Network [J]. Computer Science, 2022, 49(2): 248-255.
[15] ZHANG Hu, BAI Ping. Graph Convolutional Networks with Long-distance Words Dependency in Sentences for Short Text Classification [J]. Computer Science, 2022, 49(2): 279-284.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.