Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 240400018-7.doi: 10.11896/jsjkx.240400018

• Intelligent Computing • Previous Articles     Next Articles

Aspect-based Sentiment Analysis Based on BERT Model and Graph Attention Network

LIN Huang, LI Bicheng   

  1. College of Computer Science and Technology,Huaqiao University,Xiamen,Fujian 361021,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:LIN Huang,born in 2000,postgraduate.His main research interests include sentiment analysis in natural language processing,intelligent data management and analysis,and deep learning.
    LI Bicheng,born in 1970,professor,Ph.D supervisor.His main research interests include artificial intelligence,natural language processing,public opinion monitoring and guidance.
  • Supported by:
    Joint Fund of Equipment Pre-research and Ministry of Education(8091B022150).

Abstract: Aspect-based sentiment analysis is a fine-grained sentiment analysis task that aims to analyze the sentiment polarity of specific aspects in a given text.Current syntax-based methods heavily rely on a single parsing result from the dependency tree,and most of the existing research lacks sufficient integration of semantic and syntactic features.Therefore,this paper proposes an aspect-based sentiment analysis approach based on the BERT model and graph attention network.This method can effectively explo the semantic and syntactic information in sentence structures and fuse these information through an interactive attention mechanism to obtain more accurate sentiment features.Firstly,the BERT pre-trained model is utilized to obtain the initial vectors of the text,and an attention mechanism is employed to associate the globalsemantic information of aspect words,resulting in the semantic features of the text.Secondly,a syntax parser is used to construct a phrase structure graph and a dependency graph,and a graph attention network is applied to encode the node information,leading to the syntactic features of the text.Finally,through an interactive attention mechanism,the learned semantic and syntactic features are combined to achieve a comprehensive understanding of aspect-opinion relationships from multiple perspectives.Experimental results show that the proposed method outperforms the existing state-of-the-art methods with ACC and F1 values on multiple datasets.

Key words: Aspect-based sentiment analysis, Syntax analysis, Graph attention network, Attention mechanism

CLC Number: 

  • TP391.1
[1]WANG Y,HUANG M,ZHU X,et al.Attentionbased LSTM for aspect-level sentiment classification[C]//Conference on Empirical Methods in Natural Language Processing.2016:606-615.
[2]TANG D,QIN B,FENG X,et al.Effective lstms for target-dependent sentiment classification[C]//International Conference on Computational Linguistics.2016:3298-3307.
[3]ZENG X Q,HUA X,LIU P S,et al.Emotion Wheel and Lexicon based emotion distribution Label Enhancement Method[J].Chinese Journal of Computers,2021,44:1080-1094.
[4]WU H,ZHANG Z,SHI S,et al.Phrase dependency relational graph attention network for Aspect-based Sentiment Analysis[J].Knowledge-Based Systems,2022,236:247.
[5]WU H,ZHANG Z,WU Q.Exploring syntactic and semanticfeatures for authorship attribution[J].Applied Soft Computing,2021,111:127.
[6]ZHANG M,QIAN T.Convolution over hierarchical syntacticand lexical graphs for aspect level sentiment analysis[C]//Conference on Empirical Methods in Natural Language Processing.2020:3540-3549.
[7]WANG K,SHEN W,YANG Y,et al.Relational graph attention network for aspect-based sentiment analysis[C]//Association for Computational Linguistics.2020:3229-3238.
[8]ZHAO G Y,LV C G,FU G H,et al.Domain Specific Sentiment Words Based Attention Model for Cross-Domain Attribute-Oriented Sentiment Analysis[J].Journal of Chinese Information Processing,2021,35(6):93-102.
[9]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTM for aspect-level sentiment classification[C]//Conference on Empirical Methods in Natural Language Processing.2016:606-615.
[10]CHEN P,SUN Z,BING L,et al.Recurrent attention network on memory for aspect sentiment analysis[C]//Conference on Empirical Methods in Natural Language Processing.2017:452-461.
[11]MA D,LI S,ZHANG X,et al.Interactive attention networks foraspect-level sentiment classification[C]//Conference on International Joint Conference on Artificial Intelligence.2017:4068-4074.
[12]LU Q,ZHU Z,ZHANG D,et al.Interactive rule attention network for aspect-level sentiment analysis[J].IEEE Access,2020,8:52505-52516.
[13]XU H,LIU B,SHU L,et al.BERT post-training for review reading comprehension and aspect-based sentiment analysis[C]//Annual Conference of the North American Chapter of the Association for Computational Linguistics.2019:2324-2335.
[14]SONG Y,WANG J,JIANG T,et al.Attentional encoder network for targeted sentiment classification[C]//International Conference on Artificial Neural Networks.2019:93-103.
[15]ZHANG K,ZHANG K,ZHANG M,et al.Incorporating dynamicsemantics into pre-trained language model for aspect-based sentiment analysis[C]//Annual Meeting of the Association for Computational Linguistics.2022:3599-3610.
[16]YAN H,DAI J,QIU X,et al.A unified generative framework for aspect-based sentiment analysis[C]//International Joint Conference on Natural Language Processing.2021:2416-2429.
[17]ZHANG C,LI Q,SONG D.Aspect-based sentiment classifica-tion with aspect-specific graph convolutional networks[C]//Conference on Empirical Methods in Natural Language Proces-sing.2019:4568-4578.
[18]LI R,CHEN H,FENG F,et al.Dual graph convolutional networks for aspect-based sentiment analysis[C]//International Joint Conference on Natural Language Processing.2021:6319-6329.
[19]LIANG B,SU H,GUI L,et al.Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks[J].Knowledge-Based Systems,2022,235:1-11.
[20]CHENC,TENG Z,WANG Z,et al.Discrete opinion tree induction for aspect-based sentiment analysis[C]//Annual Meeting of the Association for Computational Linguistics.2022:2051-2064.
[21]ZHAO X,PENG H,DAI Q,et al.Rdgcn:Reinforced dependency graph convolutional network for aspect-based sentiment analysis[C]//International Conference on Web Search and Data Mi-ning.2024:976-984.
[22]ZHAO X,ZHOU Y,XU X,et al.Extensible Multi-GranularityFusion Network for Aspect-based Sentiment Analysis[J].ar-Xiv:2402.07787,2024.
[23]TU K W,LI J.A Survey of Recent Developments in Syntactic Parsing[J].Journal of Chinese Information Processing,2020,34(7):30-41.
[24]CHAI W.A Review of the Constituency Parsing[J].Computer Knowledge and Technology,2020,16(16):3-10.
[25]MRINI K,DERNONCOURT F,BUI T,et al.Rethinking self-attention:An interpretable self-attentive encoder-decoder parser[C]//Conference on Empirical Methods in Natural Language Processing.2020:731-742.
[26]HERCIG T,BRYCHCÍN T,SVOBODA L,et al.Uwb at semeval-2016 task 5:Aspect based sentiment analysis[C]//International Workshop on Semantic Evaluation.2016:342-349.
[27]DONG L,WEI F,TAN C,et al.Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Annual Meeting of the Association for Computational Linguistics(volume 2:Short papers).2014:49-54.
[28]JIANG Q,CHEN L,XU R,et al.A challenge dataset and effective models for aspect-based sentiment analysis[C]//Conference on Empirical Methods in Natural Language Processing.2019:6280-6285.
[29]FAN F,FENG Y,ZHAO D.Multi-grained attention network for aspect-level sentiment classification[C]//Conference on Empirical Methods in Natural Language Processing.2018:3433-3442.
[30]CHEN P,SUN Z,BING L,et al.Recurrent attention network on memory for aspect sentiment analysis[C]//Conference on Empirical Methods in Natural Language Processing.2017:452-461.
[31]SUN K,ZHANG R,MENSAH S,et al.Aspect-level sentiment analysis via convolution over dependency tree[C]//Conference on Empirical Methods in Natural Language Processing.2019:5679-5688.
[32]MA M,NA S,WANG H.AEGCN:An autoencoder-constrained graph convolutional network[J].Neurocomputing,2021,432:21-31.
[33]ZENG J,LIU T,JIA W,et al.Relation construction for aspect-level sentiment classification[J].Information Sciences,2022,586:209-223.
[1] LI Yunchen, ZHANG Rui, WANG Jiabao, LI Yang, WANG Ziqi, CHEN Yao. Re-parameterization Enhanced Dual-modal Realtime Object Detection Model [J]. Computer Science, 2024, 51(9): 162-172.
[2] HU Pengfei, WANG Youguo, ZHAI Qiqing, YAN Jun, BAI Quan. Night Vehicle Detection Algorithm Based on YOLOv5s and Bistable Stochastic Resonance [J]. Computer Science, 2024, 51(9): 173-181.
[3] ZHANG Tianzhi, ZHOU Gang, LIU Hongbo, LIU Shuo, CHEN Jing. Text-Image Gated Fusion Mechanism for Multimodal Aspect-based Sentiment Analysis [J]. Computer Science, 2024, 51(9): 242-249.
[4] LIU Qian, BAI Zhihao, CHENG Chunling, GUI Yaocheng. Image-Text Sentiment Classification Model Based on Multi-scale Cross-modal Feature Fusion [J]. Computer Science, 2024, 51(9): 258-264.
[5] LI Zhe, LIU Yiyang, WANG Ke, YANG Jie, LI Yafei, XU Mingliang. Real-time Prediction Model of Carrier Aircraft Landing Trajectory Based on Stagewise Autoencoders and Attention Mechanism [J]. Computer Science, 2024, 51(9): 273-282.
[6] LIU Qilong, LI Bicheng, HUANG Zhiyong. CCSD:Topic-oriented Sarcasm Detection [J]. Computer Science, 2024, 51(9): 310-318.
[7] YAO Yao, YANG Jibin, ZHANG Xiongwei, LI Yihao, SONG Gongkunkun. CLU-Net Speech Enhancement Network for Radio Communication [J]. Computer Science, 2024, 51(9): 338-345.
[8] LIU Sichun, WANG Xiaoping, PEI Xilong, LUO Hangyu. Scene Segmentation Model Based on Dual Learning [J]. Computer Science, 2024, 51(8): 133-142.
[9] ZHANG Rui, WANG Ziqi, LI Yang, WANG Jiabao, CHEN Yao. Task-aware Few-shot SAR Image Classification Method Based on Multi-scale Attention Mechanism [J]. Computer Science, 2024, 51(8): 160-167.
[10] WANG Qian, HE Lang, WANG Zhanqing, HUANG Kun. Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J]. Computer Science, 2024, 51(8): 168-175.
[11] XIAO Xiao, BAI Zhengyao, LI Zekai, LIU Xuheng, DU Jiajin. Parallel Multi-scale with Attention Mechanism for Point Cloud Upsampling [J]. Computer Science, 2024, 51(8): 183-191.
[12] PU Bin, LIANG Zhengyou, SUN Yu. Monocular 3D Object Detection Based on Height-Depth Constraint and Edge Fusion [J]. Computer Science, 2024, 51(8): 192-199.
[13] ZHANG Junsan, CHENG Ming, SHEN Xiuxuan, LIU Yuxue, WANG Leiquan. Diversified Label Matrix Based Medical Image Report Generation [J]. Computer Science, 2024, 51(8): 200-208.
[14] WANG Chao, TANG Chao, WANG Wenjian, ZHANG Jing. Infrared Human Action Recognition Method Based on Multimodal Attention Network [J]. Computer Science, 2024, 51(8): 232-241.
[15] ZHANG Lu, DUAN Youxiang, LIU Juan, LU Yuxi. Chinese Geological Entity Relation Extraction Based on RoBERTa and Weighted Graph Convolutional Networks [J]. Computer Science, 2024, 51(8): 297-303.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!