计算机科学 ›› 2022, Vol. 49 ›› Issue (2): 223-230.doi: 10.11896/jsjkx.210100046

所属专题: 自然语言处理 虚拟专题

• 人工智能 • 上一篇    下一篇

基于注意力机制和BiLSTM-CRF的消极情绪意见目标抽取

丁锋, 孙晓   

  1. 合肥工业大学计算机与信息学院 合肥230601合肥工业大学情感计算与先进智能机器安徽省重点实验室 合肥230601
  • 收稿日期:2021-01-06 修回日期:2021-05-25 出版日期:2022-02-15 发布日期:2022-02-23
  • 通讯作者: 孙晓(sunx@hfut.edu.cn)
  • 作者简介:fengdf10@163.com
  • 基金资助:
    国家自然科学基金(61976078)

Negative-emotion Opinion Target Extraction Based on Attention and BiLSTM-CRF

DING Feng, SUN Xiao   

  1. School of Computer Science and Information Engineering,Hefei University of Technology,Hefei 230601,China
    Anhui Province Key Laboratory of Affective Computing and Advanced Intelligent Machine,Hefei University of Technology,Hefei 230601,China
  • Received:2021-01-06 Revised:2021-05-25 Online:2022-02-15 Published:2022-02-23
  • About author:DING Feng,born in 1996,postgraduate.His main research interests include na-tural language processing,machine learning and textual sentiment analysis.
    SUN Xiao,born in 1980,Ph.D,professor,is a member of China Computer Federation.His main research interests include affective computing,natural language processing,machine learning and human-machine interactions.
  • Supported by:
    National Natural Science Foundation of China(61976078).

摘要: 基于方面情感分析( Aspect-Based Sentiment Analysis,ABSA)是自然语言处理的热门课题,其中意见目标抽取和意见目标情感极性分类是ABSA的基本子任务之一。而很少有研究直接抽取特定情感极性的意见目标,尤其是抽取更有潜在价值的消极情绪意见目标。文中提出了一种全新的ABSA子任务--抽取消极情绪意见目标(Negative-Emotion Opinion Target Extraction,NE-OTE),并提出了基于注意力机制和单词与字符混合嵌入的BiLSTM-CRF模型(Attention-based BiLSTM-CRF with Word Embedding and Character Embedding,AB-CE),在双向长短时记忆网络(Bi-directional Long Short-Term Memory,BiLSTM)学习文本语义信息和捕获长距离双向语义依赖关系的基础上,通过注意力机制使模型更好地关注输入序列中的关键部分和捕获与意见目标及其情感倾向相关的隐含特征,最终通过CRF层预测句子级别的全局最佳标签序列,实现对消极情绪意见目标的抽取。文中基于主流ABSA任务基准数据集构建了3个NE-OTE任务数据集,并在这些数据集上进行了广泛的实验,实验结果显示,所提模型能够有效识别消极情绪意见目标,且识别效果明显优于其他基线模型,验证了所提方法的有效性。

关键词: 情感分析, 双向长短时记忆网络, 条件随机场, 消极情绪意见目标, 注意力机制

Abstract: Aspect-based sentiment analysis (ABSA) is a popular topic for natural language processing,in which opinion target extraction and sentiment polarity classification of opinion target are one of the basic subtasks of ABSA.However,few studies directly extract the opinion targets of specific emotional polarity,especially the negative emotion opinion targets with more potential value.A new ABSA subtask--negative emotion opinion target extraction (NE-OTE) is proposed,and a BiLSTM-CRF model based on attention mechanism and character and word mixture embedding (AB-CE) is proposed.On the basis of bi-directional long short-term memory (BiLSTM) learning textual semantic information and capturing long distance bi-directional semantic dependency,through the attention mechanism,the model can better pay attention to the key parts in the input sequence and capture the implied characteristics related to the opinion target and its emotional tendency.Finally,the CRF layer can be used to predict the optimal tag sequence at the sentence level,so as to extract the negative emotional opinion target.This paper builds three NE-OTE task datasets based on the mainstream ABSA task baseline datasets and conducts extensive experiments on these datasets.Experimental results show that the model proposed in this paper can effectively identify the target of negative emotional opinions,and is significantly better than other baseline models,which verifies the effectiveness of the method proposed in this paper.

Key words: Attention, BiLSTM, CRF, Negative emotion opinion target, Sentiment analysis

中图分类号: 

  • TP391
[1]NASUKAWA T,YI J.Sentiment analysis:Capturing favorability using natural language processing[C]//Proceedings of the 2nd International Conference on Knowledge Capture.2003:70-77.
[2]LIU B.Sentiment analysis and opinion mining[J].SynthesisLectures on Human Language Technologies,2012,5(1):1-167.
[3]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:606-615.
[4]TAY Y,TUAN L A,HUI S C.Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018:5956-5963.
[5]SUN C,HUANG L,QIU X.Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence[J].arXiv:1903.09588,2019.
[6]LI X,BING L,ZHANG W,et al.Exploiting BERT for end-to-end aspect-based sentiment analysis[J].arXiv:1910.00883,2019.
[7]ZHANG J,DUAN L G,LI A P,et al.Fine-grained SentimentAnalysis Based on Combination of Attention and Gated Mechanism[J].Computer Science,2021,48(8):226-233.
[8]PORIA S,CAMBRIA E,GELBUKH A.Aspect extraction foropinion mining with a deep convolutional neural network[J].Knowledge-Based Systems,2016:42-49.
[9]XU H,LIU B,SHU L,et al.Double embeddings and cnn-basedsequence labeling for aspect extraction[J].arXiv:1805.04601,2018.
[10]MNIH V,HEESS N,GRAVES A.Recurrent models of visualattention[J].Advances in Neural Information Processing Systems,2014,27:2204-2212.
[11]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[12]RUSH A M,CHOPRA S,WESTON J.A neural attention model for abstractive sentence summarization[J].arXiv:1509.00685,2015.
[13]HU M,LIU B.Mining and summarizing customer reviews[C]//Proceedings of the Tenth ACM SIGKDD International Confe-rence on Knowledge Discovery and Data Mining.2004:168-177.
[14]ZHUANG L,JING F,ZHU X Y.Movie review mining and summarization[C]//Proceedings of the 15th ACM International Conference on Information and Knowledge Management.2006:43-50.
[15]LIU P,JOTY S,MENG H.Fine-grained opinion mining with recurrent neural networks and word embeddings[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:1433-1443.
[16]PORIA S,CAMBRIA E,GELBUKH A.Aspect extraction foropinion mining with a deep convolutional neural network[J].Knowledge-Based Systems,2016(108),108:42-49.
[17]XU H,LIU B,SHU L,et al.Double embeddings and cnn-based sequence labeling for aspect extraction[J].arXiv:1805.04601,2018.
[18]LI Y,LIU T,LI D,et al.Character-based BiLSTM-CRF Incorporating POS and Dictionaries for Chinese Opinion Target Extraction[C]//Asian Conference on Machine Learning.2018:518-533.
[19]LI J,SUN A,HAN J,et al.A survey on deep learning for named entity recognition[J].IEEE Transactions on Knowledge and Data Engineering,2020,3(17):1-20.
[20]PLANK B,SØGAARD A,GOLDBERG Y.Multilingual part-of-speech tagging with bidirectional long short-term memorymo-dels and auxiliary loss[J].arXiv:1604.05529,2016.
[21]MARTINC M,ŠKRLJ B,POLLAK S.TNT-KID:Transformer-based Neural Tagger for Keyword Identification[J].arXiv:2003.09166,2020.
[22]HE Z,WANG Z,WEI W,et al.A Survey on Recent Advances in Sequence Labeling from Deep Learning Models[J].arXiv:2011.06727,2020.
[23]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[24]BAI X.Text classification based on LSTM and attention[C]//2018 Thirteenth International Conference on Digital Information Management (ICDIM).IEEE,2018:29-32.
[25]CHIU J P C,NICHOLS E.Named entity recognition with bidirectional LSTM-CNNs[J].Transactions of the Association for Computational Linguistics,2016,4:357-370.
[26]LAFFERTY J,MCCALLUM A,PEREIRA F C N.Conditional random fields:Probabilistic models for segmenting and labeling sequence data[C]//Proc. of the 18th Int. Conf. on Machine Learning.New York:ACM,2021:282-289.
[27]HUANG Z,XU W,YU K.Bidirectional LSTM-CRF models for sequence tagging[J].arXiv:1508.01991,2015.
[28]LIU S,TANG B,CHEN Q,et al.Drug name recognition:approaches and resources[J].Information,2015,6(4):790-810.
[29]XU K,BA J,KIROS R,et al.Show,attend and tell:Neuralimage caption generation with visual attention[C]//Internatio-nal Conference on Machine Learning.2015:2048-2057.
[30]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[31]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[32]PONTIKI M,PAPAGEORGIOU H,GALANIS D,et al.SemEval-2014 Task 4:Aspect Based Sentiment Analysis[C]//Proc. 8th Int.Workshop Semantic Eval(SemEval).2014:27-35.
[33]PONTIKI M,GALANIS D,PAPAGEORGIOU H,et al.Semeval-2015 task 12:Aspect based sentiment analysis[C]//Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015).2015:486-495.
[34]PORIA S,CAMBRIA E,GELBUKH A.Aspect extraction foropinion mining with a deep convolutional neural network[J].Knowledge-Based Systems,2016,108:42-49.
[35]WEI J,ZOU K.Eda:Easy data augmentation techniques forboosting performance on text classification tasks[J].arXiv:1901.11196,2019.
[36]LAMESKI P,ZDRAVEVSKI E,MINGOV R,et al.SVM pa-rameter tuning with grid search and its impact on reduction of model over-fitting[M]//Rough sets,fuzzy sets,data mining,and granular computing.Cham:Springer,2015:464-474.
[37]ALI M N A,TAN G,HUSSAIN A.Bidirectional recurrent neural network approach for Arabic named entityrecognition[J].Future Internet,2018,10(12):123-135.
[38]HUANG Z,XU W,YU K.Bidirectional LSTM-CRF models forsequence tagging[J].arXiv:1508.01991,2015.
[1] 周芳泉, 成卫青.
基于全局增强图神经网络的序列推荐
Sequence Recommendation Based on Global Enhanced Graph Neural Network
计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[2] 戴禹, 许林峰.
基于文本行匹配的跨图文本阅读方法
Cross-image Text Reading Method Based on Text Line Matching
计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032
[3] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[4] 熊丽琴, 曹雷, 赖俊, 陈希亮.
基于值分解的多智能体深度强化学习综述
Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization
计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[5] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[6] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[7] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[9] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[10] 汪鸣, 彭舰, 黄飞虎.
基于多时间尺度时空图网络的交通流量预测模型
Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction
计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188
[11] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[12] 熊罗庚, 郑尚, 邹海涛, 于化龙, 高尚.
融合双向门控循环单元和注意力机制的软件自承认技术债识别方法
Software Self-admitted Technical Debt Identification with Bidirectional Gate Recurrent Unit and Attention Mechanism
计算机科学, 2022, 49(7): 212-219. https://doi.org/10.11896/jsjkx.210500075
[13] 彭双, 伍江江, 陈浩, 杜春, 李军.
基于注意力神经网络的对地观测卫星星上自主任务规划方法
Satellite Onboard Observation Task Planning Based on Attention Neural Network
计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093
[14] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[15] 曾志贤, 曹建军, 翁年凤, 蒋国权, 徐滨.
基于注意力机制的细粒度语义关联视频-文本跨模态实体分辨
Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism
计算机科学, 2022, 49(7): 106-112. https://doi.org/10.11896/jsjkx.210500224
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!