计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220300279-7.doi: 10.11896/jsjkx.220300279

• 人工智能 • 上一篇    下一篇

基于prompt和知识增强的方面级情感分析

李阳1,2, 唐积强3, 朱俊武1, 梁明轩1,2, 高翔1,2   

  1. 1 扬州大学信息工程学院 江苏 扬州 225000;
    2 中国科学院计算技术研究所 北京 100190;
    3 国家计算机网络应急技术处理协调中心 北京 100029
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 唐积强(tjq@cert.org.cn)
  • 作者简介:(1037759173@qq.com)
  • 基金资助:
    国家242信息安全计划项目(2021A008);北京市科技新星计划交叉学科合作课题(Z191100001119014);国家重点研发计划重点专项(2017YFC1700300,2017YFB1002300);国家自然科学基金(61702234);江苏省(扬州大学)研究生科研与实践创新计划项目(SJCX21_1551)

Aspect-based Sentiment Analysis Based on Prompt and Knowledge Enhancement

LI Yang1,2, TANG Jiqiang3, ZHU Junwu1, LIANG Mingxuan1,2, GAO Xiang1,2   

  1. 1 College of Information Engineering,Yangzhou University,Yangzhou,Jiangsu 225000,China;
    2 Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China;
    3 National Computer Network Emergency Response Technical Team/Coordination Center of China,Beijing 100029,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:LI Yang,born in 1994,master.His main research interest is natural language processing.TANG Jiqiang,born in 1981,master.His main research interests include na-tural language processing and internet security.
  • Supported by:
    National 242 Information Security Program(2021A008),Beijing NOVA Program(Z191100001119014),National Key Research and Development Program of China(2017YFC1700300,2017YFB1002300),National Natural Science Foundation of China(61702234) and Postgraduate Research &Practice Innovation Program of Jiangsu Province(Yangzhou University)(SJCX21_1551).

PDF (PC)

837

摘要： 方面级情感分析是一种新兴的细粒度情感分析任务,旨在根据给定句子和方面词判断情感极性。目前广泛使用的预训练语言模型由于训练目标和方面级情感分析的目标有差异,分析结果不好。为了缓解预训练语言模型和情感分析目标的差异,prompt被引入到方面级情感分析中,采用伪标签加方面词和意见词的方式创建prompt连续模板,并使用prompt-encoder训练伪标签使其拥有语义信息;然后,使用主题图注意力机制融合关于方面词和意见词的外部知识,根据融合外部知识的隐藏向量预测由情感词典组成的候选标签词;最后,采用求和置信度分数的方式将候选标签词的概率映射到情感极性分布空间上。实验表明,该模型在SemEval 2014任务的笔记本电脑数据集和餐厅数据集上将正确率分别提高了1.53%和3.5%。

关键词: 方面级情感分析, 预训练语言模型, prompt, 情感词典, 知识增强, 深度学习

Abstract: sentiment analysis is an emerging fine-grained sentiment analysis task that aims to judge sentiment polarity based on given sentences and aspect words.Currently widely used pre-trained language models are different due to their training objectives and those of aspect-based sentiment analysis,resulting in poor analysis results.In order to alleviate the difference between the pre-trained language model and the sentiment analysis target,prompt is introduced into aspect-based sentiment analysis,using pseudo-labels plus aspect words and opinion words to create prompt continuous templates,and using prompt-encoder to train pseudo-labels to have Semantic information;then,use the topic graph attention mechanism to fuse external knowledge about aspect words and opinion words,and predict candidate label words composed of sentiment dictionaries according to the hidden vector fused with external knowledge;The probabilities of label words are mapped onto the sentiment polarity distribution space.Experiments show that the model improves the accuracy by 1.53% and 3.5% on the Laptops dataset and Restaurants dataset of the SemEval 2014 task.

Key words: Aspect-based sentiment analysis, Pretrained language model, prompt, Sentiment dictionary, Knowledge enhancement, Deep learning

中图分类号: 

  • TP391
[1]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[2]RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving language understanding by generative pre-training[J].OpenAI Blog,2018.
[3]RAFFEL C,SHAZEER N,ROBERTSA,et al.Exploring thelimits of transfer learning with a unified text-to-text transformer[J].J.Mach.Learn.Res.,2020,21(140):1-67.
[4]LIU Y,OTT M,GOYALN,et al.Roberta:A robustly optimized bert pretraining approach[J].arXiv:1907.11692,1907.
[5]JAWAHAR G,SAGOT B,SEDDAH D.What does bert learn about the structure of language?[C]//Proceedings of ACL.2019:3651-3657.
[6]YENICELIK D,SCHMIDT F,KILCHER Y.How does bertcapture semantics? a closer look at polysemous words[C]//Proceedings of BlackboxNLP.2020:156-162.
[7]HEWITT J,MANNING C D.A structural probe for findingsyntax in word representations[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers).2019:4129-4138.
[8]PETRONI F,ROCKTÄSCHEL T,LEWIS P,et al.Languagemodels as knowledge bases?[J].arXiv:1909.01066,2019.
[9]CHEN S,WANG Y,LIU J,et al.Bidirectional machine reading comprehension for aspect sentiment triplet extraction[C]//Proceedings of The AAAI Conference on Artificial Intelligence.2021:12666-12674.
[10]YAN H,DAI J,QIU X,et al.A unified generative frameworkfor aspect-based sentiment analysis[J].arXiv:2106.04300,2021.
[11]JIANG Z,XU F F,ARAKI J,et al.How can we know what language models know?[J].Transactions of the Association for Computational Linguistics,2020,8:423-438.
[12]SHIN T,RAZEGHI Y,LOGAN IV R L,et al.Autoprompt:Eliciting knowledge from language models with automatically generated prompts[J].arXiv:2010.15980,2020.
[13]SCHICK T,SCHÜTZE H.Exploiting cloze questions for few shot text classification and natural language inference[J].ar-Xiv:2001.07676,2020.
[14]HAN X,ZHAO W,DING N,et al.Ptr:Prompt tuning withrules for text classification[J].arXiv:2105.11259,2021.
[15]LIU X,ZHENG Y,DU Z,et al.GPT understands,too[J].ar-Xiv:2103.10385,2021.
[16]DING X,LIU B,YU P S.A holistic lexicon-based approach to opinion mining[C]//Proceedings of the 2008 International Conference on Web Search and Data Mining.2008:231-240.
[17]NGUYEN T H,SHIRAI K.Aspect-based sentiment analysis using tree kernel based relation extraction[C]//International Conference on Intelligent Text Processing and Computational Linguistics.Cham:Springer,2015:114-125.
[18]LIPENKOVA J.A system for fine-grained aspect-based sentiment analysis of Chinese[C]//Proceedings of ACL-IJCNLP 2015 System Demonstrations.2015:55-60.
[19]KIRITCHENKO S,ZHU X,CHERRY C,et al.Detecting as-pects and sentiment in customer reviews[C]//8th International Workshop on Semantic Evaluation(SemEval).2014:437-442.
[20]RAO D,RAVICHANDRAN D.Semi-supervised polarity lexicon induction[C]//Proceedings of the 12th Conference of the European Chapter of the ACL(EACL 2009).2009:675-682.
[21]SUN K,ZHANG R,MENSAH S,et al.Aspect-level sentiment analysis via convolution over dependency tree[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).2019:5679-5688.
[22]WANG K,SHEN W,YANG Y,et al.Relational Graph Attention Network for Aspect-based Sentiment Analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:3229-3238.
[23]TANG J,LU Z,SU J,et al.Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:557-566.
[24]HE R,LEE W S,NGH T,et al.An Interactive Multi-TaskLearning Network for End-to-End Aspect-Based Sentiment Analysis[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:504-515.
[25]WU C,XIONG Q,YI H,et al.Multiple-element joint detection for Aspect-Based Sentiment Analysis[J].Knowledge-Based Systems,2021,223:107073.
[26]LI Y,YIN C,ZHONG S,et al.Multi-Instance Multi-LabelLearning Networks for Aspect-Category Sentiment Analysis[C]//Proceedings of the 2020 Conference on Empirical Me-thods in Natural Language Processing(EMNLP).2020:3550-3560.
[27]CHEN Z,QIAN T.Relation-aware collaborative learning for unified aspect-based sentiment analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:3685-3694.
[28]BROWN T,MANN B,RYDER N,et al.Language models arefew-shot learners[J].Advances in Neural Information Proces-sing Systems,2020,33:1877-1901.
[29]PETRONI F,ROCKTÄSCHEL T,LEWISP,et al.Languagemodels as knowledge bases?[J].arXiv:1909.01066,2019.
[30]DAVISON J,FELDMAN J,RUSH A M.Commonsense know-ledge mining from pretrained models[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).2019:1173-1178.
[31]HAN X,ZHAO W,DING N,et al.Ptr:Prompt tuning withrules for text classification[J].arXiv:2105.11259,2021.
[32]LIU X,ZHENG Y,DU Z,et al.GPT understands,too[J].ar-Xiv:2103.10385,2021.
[33]LI X L,LIANGP.Prefix-tuning:Optimizing continuous prompts for generation[J].arXiv:2101.00190,2021.
[34]SCHICK T,SCHMID H,SCHÜTZE H.Automatically identifying words that can serve as labels for few-shot text classification[J].arXiv:2010.13641,2020.
[35]BENGIO Y.From system 1 deep learning to system 2 deeplearning[C]//Neural Information Processing Systems.2019.
[36]PENG H,XU L,BING L,et al.Knowing what,how and why:A near complete solution for aspect-based sentiment analysis[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:8600-8607.
[37]TANG D,QIN B,FENG X,et al.Effective LSTMs for target-dependent sentiment classification[J].arXiv:1512.01100,2015.
[38]DONG L,WEI F,TAN C,et al.Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics(volume 2:Short papers).2014:49-54.
[39]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTMfor aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:606-615.
[40]MA D,LI S,ZHANG X,et al.Interactive attention networks for aspect-level sentiment classification[J].arXiv:1709.00893,2017.
[41]TANG D,QIN B,LIU T.Aspect level sentiment classificationwith deep memory network[J].arXiv:1605.08900,2016.
[42]CHEN P,SUN Z,BING L,et al.Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.2017:452-461.
[43]LIU Q,ZHANG H,ZENG Y,et al.Content attention model for aspect based sentiment analysis[C]//Proceedings of the 2018 World Wide Web Conference.2018:1023-1032.
[44]DENG L M,WEI J J,WU Y B,et al.Based on knowledge map and circular attention network Perspective Level Emotional Analysis[J].Pattern Recognition and Artificial Intelligence,2020,33(6):9.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发