Computer Science ›› 2023, Vol. 50 ›› Issue (11A): 220900266-6.doi: 10.11896/jsjkx.220900266

• Artificial Intelligence • Previous Articles     Next Articles

Aspect-based Sentiment Analysis Based on Local Context Focus Mechanism and Talking-Head Attention

LIN Zhengchao, LI Bicheng   

  1. College of Computer Science and Technology,Huaqiao University,Xiamen,Fjuian 361021,China
  • Published:2023-11-09
  • About author:LIN Zhengchao,born in 1997,postgra-duate.His main research interests include natural language processing,intelligent data management and analysis,and deep learning.
    LI Bicheng,born in 1970,Ph.D,professor,Ph.D supervisor.His main research interests include artificial intelligence,natural language processing,online public opinion monitoring and guidance.
  • Supported by:
    Joint Fund Project of Ministry of Education for Equipment Pre-research(8091B022150).

Abstract: Aspect-based sentiment analysis is an important research direction in the field of natural language processing,and its purpose is to predict the sentiment polarity of different aspects in sentences.The existing aspect-based sentiment analysis usually ignores the relationship between sentiment polarity and local context,and the operation of each attention head in the multi-head attention used is independent of each other.To this end,an aspect-based sentiment analysis model based on the localcontext focus mechanism and talking-head attention is proposed.First,preliminary features of local context and global context are captured by a BERT pretrained model.Then in the feature extraction layer,the local contextualfocus mechanism is used,and the local contextual features are further extracted through the contextual feature dynamic mask layer combined with the talking-head attention,and talking-head attention is used to further extract global context features.Finally,the local and global information are fused and input to the nonlinear layer to obtain sentiment analysis results.Experiments are conducted on three public datasets.Experiments show that compared with multiple existing baseline models,the MF1 value and accuracy of the new model are improved.

Key words: Aspect-based sentiment analysis, BERT model, Local context focus mechanism, Talking-Head attention

CLC Number: 

  • TP391
[1]ZHANG L,WANG S,LIU B.Deep learning for sentiment ana-lysis:A survey[J].Wiley Interdisciplinary Reviews:Data Mining and Knowledge Discovery,2018,8(4):e1253.
[2]LV Y,WEI F,CAO L,et al.Aspect-level sentiment analysisusing context and aspect memory network[J].Neurocomputing,2021,428:195-205.
[3]ZHOU J,HUANG J X,CHEN Q,et al.Deep Learning for Aspect-Level Sentiment Classification:Survey,Vision and Challenges[J].IEEE Access,2019,7:78454-78483.
[4]LIU B.Sentiment analysis:Mining opinions,sentiments,andemotions [J].Computational Linguistics,2016,42(3):1-4.
[5]RUZ G A,HENRÍQUEZ P A,MASCAREÑO A.Sentimentanalysis of Twitter data during critical events through Bayesian networks classifiers[J].Future Generation Computer Systems,2020,106:92-104.
[6]TANG D,QIN B,FENG X,et al.Effective lstms for target-dependent sentiment classification[C]//Proceedings of COLING 2016,the 26th International Conference on Computational Linguistics:Technical Papers.2016.
[7]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[8]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:606-615.
[9]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Advances in Neural Information Processing Systems.2017.
[10]MA D,LI S,ZHANG X,et al.Interactive attention networks for aspect-level sentiment classification[C]//Twenty- Sixth International Joint Conference on Artificial Intelligence.2017.
[11]FAN F,FENG Y,ZHAO D.Multi-grained attention network for aspect-level sentiment classification[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:3433-3442.
[12]SUN C,HUANG L,QIU X.Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence[J].arXiv:1903.09588,2019.
[13]SONG Y,WANG J,JIANGT,et al.Attentional encoder network for targeted sentiment classification[J].arXiv:1902.09314,2019.
[14]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[15]BHOJANAPALLI S,YUN C,RAWAT A S,et al.Low-Rank Bottleneck in Multi-head Attention Models[J].arXiv:2002.07028,2020.
[16]SHAZEER N,LAN Z,CHENG Y,et al.Talking-Heads Attention[J].arXiv:2003.02436,,2020.
[17]PONTIKI M,GALANIS D,PAPAGEORGIOU H,et al.Semeval-2016 task 5:Aspect based sentiment analysis[C]//International Workshop on Semantic Evaluation.2016:19-30.
[18]DONG L,WEI F,TAN C,et al.Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics(volume 2:Short papers).2014:49-54.
[19]CHEN P,SUN Z,BING L,et al.Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.2017:452-461.
[20]ZENG B,YANG H,XU R,et al.Lcf:A local context focusmechanism for aspect-based sentiment classification[J].Applied Sciences,2019,9(16):3389.
[21]LIANG Y,MENG F,ZHANG J,et al.A dependency syntactic knowledge augmented interactive architecture for end-to-end aspect-based sentiment analysis[J].Neurocomputing,2021,454:291-302.
[1] LIU Yingying, YANG Qiuhui, YAO Bangguo, LIU Qiaoyun. Study on REST API Test Case Generation Method Based on Dependency Model [J]. Computer Science, 2023, 50(9): 101-107.
[2] LI Yang, TANG Jiqiang, ZHU Junwu, LIANG Mingxuan, GAO Xiang. Aspect-based Sentiment Analysis Based on Prompt and Knowledge Enhancement [J]. Computer Science, 2023, 50(6A): 220300279-7.
[3] YANG Ying, ZHANG Fan, LI Tianrui. Aspect-based Sentiment Analysis Based on Dual-channel Graph Convolutional Network with Sentiment Knowledge [J]. Computer Science, 2023, 50(5): 230-237.
[4] LUO Liang, CHENG Chunling, LIU Qian, GUI Yaocheng. Answer Selection Model Based on MLP and Semantic Matrix [J]. Computer Science, 2023, 50(5): 270-276.
[5] JIA Jingdong, ZHANG Minnan, ZHAO Xiang, HUANG Jian. Study on Scheduling Algorithm of Intelligent Order Dispatching [J]. Computer Science, 2023, 50(11A): 230300029-7.
[6] DENG Ruhan, ZHANG Qinghua, HUANG Shuaishuai, GAO Man. Novel Graph Convolutional Network Based on Multi-granularity Feature Fusion for Aspect-basedSentiment Analysis [J]. Computer Science, 2023, 50(10): 80-87.
[7] HE Zhihao, CHEN Hongmei, LUO Chuan. Aspect-based Sentiment Analysis Based on Aspect Semantic and Gated Filtering Network [J]. Computer Science, 2023, 50(10): 193-202.
[8] YU Jia-qi, KANG Xiao-dong, BAI Cheng-cheng, LIU Han-qing. New Text Retrieval Model of Chinese Electronic Medical Records [J]. Computer Science, 2022, 49(6A): 32-38.
[9] PAN Fang, ZHANG Hui-bing, DONG Jun-chao, SHOU Zhao-yu. Aspect Sentiment Analysis of Chinese Online Course Review Based on Efficient Transformer [J]. Computer Science, 2021, 48(6A): 264-269.
[10] SUN Zhong-feng, WANG Jing. RCNN-BGRU-HN Network Model for Aspect-based Sentiment Analysis [J]. Computer Science, 2019, 46(9): 223-228.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!