基于prompt和知识增强的方面级情感分析

doi:10.11896/jsjkx.220300279

Abstract

Abstract: sentiment analysis is an emerging fine-grained sentiment analysis task that aims to judge sentiment polarity based on given sentences and aspect words.Currently widely used pre-trained language models are different due to their training objectives and those of aspect-based sentiment analysis,resulting in poor analysis results.In order to alleviate the difference between the pre-trained language model and the sentiment analysis target,prompt is introduced into aspect-based sentiment analysis,using pseudo-labels plus aspect words and opinion words to create prompt continuous templates,and using prompt-encoder to train pseudo-labels to have Semantic information;then,use the topic graph attention mechanism to fuse external knowledge about aspect words and opinion words,and predict candidate label words composed of sentiment dictionaries according to the hidden vector fused with external knowledge;The probabilities of label words are mapped onto the sentiment polarity distribution space.Experiments show that the model improves the accuracy by 1.53% and 3.5% on the Laptops dataset and Restaurants dataset of the SemEval 2014 task.

Key words: Aspect-based sentiment analysis, Pretrained language model, prompt, Sentiment dictionary, Knowledge enhancement, Deep learning

CLC Number:

TP391

LI Yang, TANG Jiqiang, ZHU Junwu, LIANG Mingxuan, GAO Xiang. Aspect-based Sentiment Analysis Based on Prompt and Knowledge Enhancement[J].Computer Science, 2023, 50(6A): 220300279-7.

References

[1]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[2]RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving language understanding by generative pre-training[J].OpenAI Blog,2018.
[3]RAFFEL C,SHAZEER N,ROBERTSA,et al.Exploring thelimits of transfer learning with a unified text-to-text transformer[J].J.Mach.Learn.Res.,2020,21(140):1-67.
[4]LIU Y,OTT M,GOYALN,et al.Roberta:A robustly optimized bert pretraining approach[J].arXiv:1907.11692,1907.
[5]JAWAHAR G,SAGOT B,SEDDAH D.What does bert learn about the structure of language?[C]//Proceedings of ACL.2019:3651-3657.
[6]YENICELIK D,SCHMIDT F,KILCHER Y.How does bertcapture semantics? a closer look at polysemous words[C]//Proceedings of BlackboxNLP.2020:156-162.
[7]HEWITT J,MANNING C D.A structural probe for findingsyntax in word representations[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers).2019:4129-4138.
[8]PETRONI F,ROCKTÄSCHEL T,LEWIS P,et al.Languagemodels as knowledge bases?[J].arXiv:1909.01066,2019.
[9]CHEN S,WANG Y,LIU J,et al.Bidirectional machine reading comprehension for aspect sentiment triplet extraction[C]//Proceedings of The AAAI Conference on Artificial Intelligence.2021:12666-12674.
[10]YAN H,DAI J,QIU X,et al.A unified generative frameworkfor aspect-based sentiment analysis[J].arXiv:2106.04300,2021.
[11]JIANG Z,XU F F,ARAKI J,et al.How can we know what language models know?[J].Transactions of the Association for Computational Linguistics,2020,8:423-438.
[12]SHIN T,RAZEGHI Y,LOGAN IV R L,et al.Autoprompt:Eliciting knowledge from language models with automatically generated prompts[J].arXiv:2010.15980,2020.
[13]SCHICK T,SCHÜTZE H.Exploiting cloze questions for few shot text classification and natural language inference[J].ar-Xiv:2001.07676,2020.
[14]HAN X,ZHAO W,DING N,et al.Ptr:Prompt tuning withrules for text classification[J].arXiv:2105.11259,2021.
[15]LIU X,ZHENG Y,DU Z,et al.GPT understands,too[J].ar-Xiv:2103.10385,2021.
[16]DING X,LIU B,YU P S.A holistic lexicon-based approach to opinion mining[C]//Proceedings of the 2008 International Conference on Web Search and Data Mining.2008:231-240.
[17]NGUYEN T H,SHIRAI K.Aspect-based sentiment analysis using tree kernel based relation extraction[C]//International Conference on Intelligent Text Processing and Computational Linguistics.Cham:Springer,2015:114-125.
[18]LIPENKOVA J.A system for fine-grained aspect-based sentiment analysis of Chinese[C]//Proceedings of ACL-IJCNLP 2015 System Demonstrations.2015:55-60.
[19]KIRITCHENKO S,ZHU X,CHERRY C,et al.Detecting as-pects and sentiment in customer reviews[C]//8th International Workshop on Semantic Evaluation(SemEval).2014:437-442.
[20]RAO D,RAVICHANDRAN D.Semi-supervised polarity lexicon induction[C]//Proceedings of the 12th Conference of the European Chapter of the ACL(EACL 2009).2009:675-682.
[21]SUN K,ZHANG R,MENSAH S,et al.Aspect-level sentiment analysis via convolution over dependency tree[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).2019:5679-5688.
[22]WANG K,SHEN W,YANG Y,et al.Relational Graph Attention Network for Aspect-based Sentiment Analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:3229-3238.
[23]TANG J,LU Z,SU J,et al.Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:557-566.
[24]HE R,LEE W S,NGH T,et al.An Interactive Multi-TaskLearning Network for End-to-End Aspect-Based Sentiment Analysis[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:504-515.
[25]WU C,XIONG Q,YI H,et al.Multiple-element joint detection for Aspect-Based Sentiment Analysis[J].Knowledge-Based Systems,2021,223:107073.
[26]LI Y,YIN C,ZHONG S,et al.Multi-Instance Multi-LabelLearning Networks for Aspect-Category Sentiment Analysis[C]//Proceedings of the 2020 Conference on Empirical Me-thods in Natural Language Processing(EMNLP).2020:3550-3560.
[27]CHEN Z,QIAN T.Relation-aware collaborative learning for unified aspect-based sentiment analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:3685-3694.
[28]BROWN T,MANN B,RYDER N,et al.Language models arefew-shot learners[J].Advances in Neural Information Proces-sing Systems,2020,33:1877-1901.
[29]PETRONI F,ROCKTÄSCHEL T,LEWISP,et al.Languagemodels as knowledge bases?[J].arXiv:1909.01066,2019.
[30]DAVISON J,FELDMAN J,RUSH A M.Commonsense know-ledge mining from pretrained models[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).2019:1173-1178.
[31]HAN X,ZHAO W,DING N,et al.Ptr:Prompt tuning withrules for text classification[J].arXiv:2105.11259,2021.
[32]LIU X,ZHENG Y,DU Z,et al.GPT understands,too[J].ar-Xiv:2103.10385,2021.
[33]LI X L,LIANGP.Prefix-tuning:Optimizing continuous prompts for generation[J].arXiv:2101.00190,2021.
[34]SCHICK T,SCHMID H,SCHÜTZE H.Automatically identifying words that can serve as labels for few-shot text classification[J].arXiv:2010.13641,2020.
[35]BENGIO Y.From system 1 deep learning to system 2 deeplearning[C]//Neural Information Processing Systems.2019.
[36]PENG H,XU L,BING L,et al.Knowing what,how and why:A near complete solution for aspect-based sentiment analysis[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:8600-8607.
[37]TANG D,QIN B,FENG X,et al.Effective LSTMs for target-dependent sentiment classification[J].arXiv:1512.01100,2015.
[38]DONG L,WEI F,TAN C,et al.Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics(volume 2:Short papers).2014:49-54.
[39]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTMfor aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:606-615.
[40]MA D,LI S,ZHANG X,et al.Interactive attention networks for aspect-level sentiment classification[J].arXiv:1709.00893,2017.
[41]TANG D,QIN B,LIU T.Aspect level sentiment classificationwith deep memory network[J].arXiv:1605.08900,2016.
[42]CHEN P,SUN Z,BING L,et al.Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.2017:452-461.
[43]LIU Q,ZHANG H,ZENG Y,et al.Content attention model for aspect based sentiment analysis[C]//Proceedings of the 2018 World Wide Web Conference.2018:1023-1032.
[44]DENG L M,WEI J J,WU Y B,et al.Based on knowledge map and circular attention network Perspective Level Emotional Analysis[J].Pattern Recognition and Artificial Intelligence,2020,33(6):9.

Related Articles 15

[1]	WANG Yu, WANG Zuchao, PAN Rui. Survey of DGA Domain Name Detection Based on Character Feature [J]. Computer Science, 2023, 50(8): 251-259.
[2]	ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[3]	SONG Xinyang, YAN Zhiyuan, SUN Muyi, DAI Linlin, LI Qi, SUN Zhenan. Review of Talking Face Generation [J]. Computer Science, 2023, 50(8): 68-78.
[4]	WANG Xu, WU Yanxia, ZHANG Xue, HONG Ruize, LI Guangsheng. Survey of Rotating Object Detection Research in Computer Vision [J]. Computer Science, 2023, 50(8): 79-92.
[5]	ZHOU Ziyi, XIONG Hailing. Image Captioning Optimization Strategy Based on Deep Learning [J]. Computer Science, 2023, 50(8): 99-110.
[6]	ZHANG Xiao, DONG Hongbin. Lightweight Multi-view Stereo Integrating Coarse Cost Volume and Bilateral Grid [J]. Computer Science, 2023, 50(8): 125-132.
[7]	LI Kun, GUO Wei, ZHANG Fan, DU Jiayu, YANG Meiyue. Adversarial Malware Generation Method Based on Genetic Algorithm [J]. Computer Science, 2023, 50(7): 325-331.
[8]	WANG Mingxia, XIONG Yun. Disease Diagnosis Prediction Algorithm Based on Contrastive Learning [J]. Computer Science, 2023, 50(7): 46-52.
[9]	SHEN Zhehui, WANG Kailai, KONG Xiangjie. Exploring Station Spatio-Temporal Mobility Pattern:A Short and Long-term Traffic Prediction Framework [J]. Computer Science, 2023, 50(7): 98-106.
[10]	HUO Weile, JING Tao, REN Shuang. Review of 3D Object Detection for Autonomous Driving [J]. Computer Science, 2023, 50(7): 107-118.
[11]	ZHOU Bo, JIANG Peifeng, DUAN Chang, LUO Yuetong. Study on Single Background Object Detection Oriented Improved-RetinaNet Model and Its Application [J]. Computer Science, 2023, 50(7): 137-142.
[12]	MAO Huihui, ZHAO Xiaole, DU Shengdong, TENG Fei, LI Tianrui. Short-term Subway Passenger Flow Forecasting Based on Graphical Embedding of Temporal Knowledge [J]. Computer Science, 2023, 50(7): 213-220.
[13]	LI Yuqiang, LI Linfeng, ZHU Hao, HOU Mengshu. Deep Learning-based Algorithm for Active IPv6 Address Prediction [J]. Computer Science, 2023, 50(7): 261-269.
[14]	LIANG Mingxuan, WANG Shi, ZHU Junwu, LI Yang, GAO Xiang, JIAO Zhixiang. Survey of Knowledge-enhanced Natural Language Generation Research [J]. Computer Science, 2023, 50(6A): 220200120-8.
[15]	WANG Dongli, YANG Shan, OUYANG Wanli, LI Baopu, ZHOU Yan. Explainability of Artificial Intelligence:Development and Application [J]. Computer Science, 2023, 50(6A): 220600212-7.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Aspect-based Sentiment Analysis Based on Prompt and Knowledge Enhancement

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0