计算机科学 ›› 2022, Vol. 49 ›› Issue (3): 218-224.doi: 10.11896/jsjkx.210400034

• 人工智能 • 上一篇    下一篇

跨领域文本的可迁移情绪分析方法

张舒萌1, 余增1, 李天瑞1,2   

  1. 1 西南交通大学计算机与人工智能学院 成都611756
    2 综合交通大数据应用技术国家工程实验室 成都611756
  • 收稿日期:2021-04-02 修回日期:2021-07-28 出版日期:2022-03-15 发布日期:2022-03-15
  • 通讯作者: 李天瑞(trli@swjtu.edu.cn)
  • 作者简介:(zzshumeng@163.com)
  • 基金资助:
    国家重点研发课题(2020AAA0105101)

Transferable Emotion Analysis Method for Cross-domain Text

ZHANG Shu-meng1, YU Zeng1, LI Tian-rui1,2   

  1. 1 School of Computing and Artificial Intelligence,Southwest Jiaotong University,Chengdu 611756,China
    2 National Engineering Laboratory of Integrated Transportation Big Data Application Technology,Chengdu 611756,China
  • Received:2021-04-02 Revised:2021-07-28 Online:2022-03-15 Published:2022-03-15
  • About author:ZHANG Shu-meng,born in 1996,postgraduate.Her main research interests include natural language processing and sentiment analysis.
    LI Tian-rui,born in 1969,Ph.D,professor,Ph.D supervisor,is a distinguished member of China Computer Federation.His main research interests include big data intelligence,rough sets and granular computing.
  • Supported by:
    National Key R & D Program of China(2020AAA0105101).

摘要: 随着移动互联网的迅猛发展,社交网络平台充斥着大量带有情绪色彩的文本数据,对此类文本中的情绪进行分析研究不仅有助于了解网民的态度和情感,而且对科研机构和政府掌握社会的情绪变化及走向有着重要作用。传统的情感分析主要对情感倾向进行分析,无法精确、多维度地描述出文本的情绪,为了解决这个问题,文中对文本的情绪分析进行研究。首先针对不同领域文本数据集中情绪标签缺乏的问题,提出了一个基于深度学习的可迁移情绪分类的情感分析模型FMRo-BLA,该模型对通用领域文本进行预训练,然后通过基于参数的迁移学习、特征融合和FGM对抗学习,将预训练模型应用于特定领域的下游情感分析任务中,最后在微博的公开数据集上进行对比实验。结果表明,该方法相比于目前性能最好的RoBERTa预训练语言模型,在目标领域数据集上F1值有5.93%的提升,进一步加入迁移学习后F1值有12.38%的提升。

关键词: 迁移学习, 情绪分析, 深度学习, 特征融合

Abstract: With the rapid development of mobile internet,social network platform is full of a large number of text data with emotional color.Mining the emotion in such text not only helps to understand the attitude and emotion of internet users,but also plays an important role in scientific research institutions and the government to grasp the emotional changes and trends of society.Traditional sentiment analysis mainly focuses on the analysis of sentiment tendency,which can not accurately and multi-dimensionally describe the emotion of the text.In order to solve this problem,this paper studies the emotion analysis of the text.Firstly,aiming at the lack of fine-grained sentiment tags in text data sets of different fields,a deep learning based emotion analysis model,FMRo-BLA,is proposed.The model pre-trains the general domain text,and then applies the pre-trained model to the downstream situation of specific domain through parameter based migration learning,feature fusion and FGM Adversarial training.Compared with the best performance of RoBERTa pre-trained language model,the F1 value of the proposed method is improved by 5.93% on the target domain dataset,and it achieves 12.38% by further adding transfer learning.

Key words: Deep learning, Emotion analysis, Feature fusion, Transfer learning

中图分类号: 

  • TP391
[1]REBUFFI S A,BILEN H,VEDALDI A.Learning multiple vi-sual domains with residual adapters[C]//Proceedings of the 31st Neural Information Processing Systems.2017:506-516.
[2]YOSINSKI J,CLUNE J,BENGIO Y,et al.How transferableare features in deep neural networks?[J].Eprint Arxiv,2014,27:3320-3328.
[3]HOWARD J,RUDER S.Universal language model fine-tuning for text classification[J].arXiv:1801.06146,2018.
[4]MIYATO T,DAI A M,GOODFELLOW I.Adversarial training methods for semi-supervised text classification[J].arXiv:1605.07725,2016.
[5]YANG M,ZHU D,CHOW K P.A topic model for building fine-grained domain-specific emotion lexicon[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2:Short Papers).2014:421-426.
[6]XU J,XU R,ZHENG Y,et al.Chinese emotion lexicon developing via multi-lingual lexical resources integration[C]//International Conference on Intelligent Text Processing and Computational Linguistics.Berlin:Springer,2013:174-182.
[7]AMAN S,SZPAKOWICZ S.Identifying expressions of emotion in text[C]//International Conference on Text,Speech and Dialogue.Berlin:Springer,2007:196-205.
[8]LI D,CAO F Y,CAO Y D,et al.Text Sentiment ClassificationBased on Phrase Patterns[J].Computer Science,2008,35(4):132-134.
[9]LEE S Y M,CHEN Y,HUANG C R.A text-driven rule-based system for emotion causedetection[C]//Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text.2010:45-53.
[10]LI W,XU H.Text-based emotion classification using emotion cause extraction[J].Expert Systems with Applications,2014,41(4):1742-1749.
[11]GAO K,XU H,WANG J.A rule-based approach to emotion cause detection for Chinese micro-blogs[J].Expert Systems with Applications,2015,42(9):4517-4528.
[12]ZHAO J,DONG L,WU J,et al.Moodlens:an emoticon-basedsentiment analysis system for Chinese tweets[C]//Proceedings of the 18th ACM SIGKDD International Conference on Know-ledge Discovery and Data Mining.2012:1528-1531.
[13]KANG X,REN F,WU Y.Bottom up:Exploring word emotions for Chinese sentence chief sentiment classification[C]//Procee-dings of the 6th International Conference on Natural Language Processing and Knowledge Engineering (NLPKE-2010).IEEE,2010:1-5.
[14]ZHANG X,LI W,LU S.Emotion detection in online social network based on multi-label learning[C]//International Confe-rence on Database Systems for Advanced Applications.Cham:Springer,2017:659-674.
[15]SUN X,LI C,YE J.Chinese microblogging emotion classifica-tion based on support vector machine[C]//Fifth International Conference on Computing,Communications and Networking Technologies (ICCCNT).IEEE,2014:1-5.
[16]SINTSOVA V,MUSAT C,PU P.Semi-supervised method for multi-category emotion recognition in tweets[C]//Proceedings of the 14th International Conference on Data MiningWorkshop.IEEE,2014:393-402.
[17]SUTTLES J,IDE N.Distant supervision for emotion classification with discrete binary values[C]//International Conference on Intelligent Text Processing and Computational Linguistics.Berlin:Springer,2013:121-136.
[18]JIANG F,LIU Y Q,LUAN H B,et al.Microblog sentimentanalysis with emoticon space model[J].Journal of Computer Science and Technology,2015,30(5):1120-1129.
[19]OUYANG X,ZHOU P,LI C H,et al.Sentiment analysis using convolutional neural network[C]//Proceedings of the 5th International Conference on Computer and Information Technology;Ubiquitous Computing and Communications;Dependable,Autonomic and Secure Computing;Pervasive Intelligence and Computing.IEEE,2015:2359-2364.
[20]SANTOS C D,GATTI M.Deep convolutional neural networks for sentiment analysis of short texts[C]//Proceedings of the 25th International Conference on Computational Linguistics:Technical Papers.2014:69-78.
[21]KIM Y.Convolutional Neural Networks for Sentence Classification[J].arXiv:1408.5882,2014.
[22]CHIU J P C,NICHOLS E.Named entity recognition with bidirectional LSTM-CNNs[J].Transactions of the Association for Computational Linguistics,2016,4:357-370.
[23]CHEN K,LIANG B,KE W D,et al.Chinese Micro-Blog Sentiment Analysis Based on Multi-Channels Convolutional Neural Networks[J].Journal of Computer Research and Development,2018,55(5):945-957.
[24]IRSOY O,CARDIE C.Deep recursive neural networks for compositionality in language[J].Advances in Neural Information Processing Systems,2014,27:2096-2104.
[25]ZHU X,SOBIHANI P,GUO H.Long short-term memory over recursive structures[C]//Proceedings of the 32th International Conference on Machine Learning.PMLR,2015:1604-1612.
[26]BRAHMA S.Improved sentence modeling using suffix bidirectional LSTM[J].arXiv:1805.07340,2018.
[27]DU Y,HE M,WANG L,et al.Wasserstein based transfer network for cross-domain sentiment classification[J].Knowledge-Based Systems,2020,204:106162.
[28]LIU J,ZHENG S,XU G,et al.Cross-domain sentiment aware word embeddings for review sentiment analysis[J].Internatio-nal Journal of Machine Learning and Cybernetics,2020,12:343-354.
[29]BLITZER J,DREDZE M,PEREIRA F.Biographies,bollywood,boom-boxes and blenders:Domain adaptation for sentiment classification[C]//Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics.2007:440-447.
[30]PAN S J,NI X,SUN J T,et al.Cross-domain sentiment classification via spectral feature alignment[C]//Proceedings of the 19th International Conference on World Wide Web.2010:751-760.
[31]DU C,SUN H,WANG J,et al.Adversarial and domain-aware bert for cross-domain sentiment analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:4019-4028.
[32]BORGWARDT K M,GRETTON A,RASCH M J,et al.In-tegrating structured biological data by kernelmaximum mean discrepancy[J].Bioinformatics,2006,22(14):49-57.
[33]PENG M,ZHANG Q,JIANG Y,et al.Cross-domain sentiment classification with target domain specific information[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics,2018:2505-2513.
[34]YANG M,YIN W,QU Q,et al.Neural attentive network forcross-domain aspect-level sentiment classification[C]//IEEE Transactions on Affective Computing.2019:761-775.
[35]ZHANG Y,MIAO D,WANG J.Hierarchical attentiongenerative adversarial networks for cross-domain sentiment classification[J].arXiv:1903.11334,2019.
[36]JI J,LUO C,CHEN X,et al.Cross-domain sentiment classification via a bifurcated-LSTM[C]//Pacific-Asia Conference on Knowledge Discovery and DataMining.Cham:Springer,2018:681-693.
[37]JAWAHAR G,SAGOT B,SEDDAH D.What does BERT learn about the structure of language?[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:3651-3657.
[38]CHEN X,QIU X,ZHU C.Bi-directional Long short-term memory neural networks for chinese word segmentation[C]//Proceedings of the 21st Conference on Empirical Methods in Natural Language Processing.2015:1197-1206.
[39]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[40]LIU Y,OTT M,GOYAL N,et al.RoBERTa:A robustly optimized bert pretraining approach[J].arXiv:1907.11692,2019.
[1] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[2] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[3] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[4] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7] 方义秋, 张震坤, 葛君伟.
基于自注意力机制和迁移学习的跨领域推荐算法
Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning
计算机科学, 2022, 49(8): 70-77. https://doi.org/10.11896/jsjkx.210600011
[8] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[9] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[10] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[11] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[12] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[13] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[14] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[15] 郁舒昊, 周辉, 叶春杨, 王太正.
SDFA:基于多特征融合的船舶轨迹聚类方法研究
SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion
计算机科学, 2022, 49(6A): 256-260. https://doi.org/10.11896/jsjkx.211100253
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!