Computer Science ›› 2025, Vol. 52 ›› Issue (10): 247-257.doi: 10.11896/jsjkx.240800061

• Artificial Intelligence • Previous Articles     Next Articles

Text Sentiment Classification Method Based on Large-batch Adversarial Strategy and EnhancedFeature Extraction

CHEN Jiahao1, DUAN Liguo1,2, CHANG Xuanwei1, LI Aiping1, CUI Juanjuan1, HAO Yuanbin1   

  1. 1 College of Computer Science and Technology,Taiyuan University of Technology,Taiyuan 030024,China
    2 Shanxi Electronic Science and Technology Institute,Linfen,Shanxi 041000,China
  • Received:2024-08-12 Revised:2024-11-30 Online:2025-10-15 Published:2025-10-14
  • About author:DUAN Liguo,born in 1970,is a member of CCF(No.15823S).His main research interest is natural language processing.
  • Supported by:
    Natural Science Foundation of Shanxi Province,China(202203021221234,202303021211052).

Abstract: The text sentiment classification task aims to analyze short text sentences and determine their corresponding sentiment categories.In order to solve the problems of lack of large-scale high-quality corpus dataset and insufficient non-uniform importance extraction of text features in the existing models in sentiment classification,this paper proposes a text sentiment classification method based on large-batch adversarial strategy and enhanced feature extraction.Firstly,the text dataset is input into the pre-trained language model BERT to obtain the corresponding word embedding vector representation,and then the BiLSTM is used to further learn the context dependencies in the sequence.Then,the local attention mechanism is combined with the local receptive field weighting of TextCNN to enhance the feature extraction ability.Finally,the output of BiLSTM and the output of TextCNN are spliced to obtain the deep feature fusion of the two spaces,which are handed over to the classifier for the judgment of sentiment classification.In the whole training process,a large-batch adversarial strategy is adopted,and adversarial perturbations are added to the word embedding space and multiple iterations are carried out to improve the robustness of the model.Experimental results on multiple datasets verify the effectiveness of the proposed model.

Key words: Short text,Sentiment classification,Adversarial strategy,Feature extraction,Word embeddings

CLC Number: 

  • TP391
[1]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.ACL,2019:4171-4186.
[2]ZHAO W M,ALWIDIAN S A,MAHMOUD Q H.Adversarial Training Methods for Deep Learning:A Systematic Review[J].Algorithms,2022,15(8):283.
[3]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[4]HUANG Z H,XU W,YU K.Bidirectional LSTM-CRF Models for Sequence Tagging[J].arXiv:1508.01991,2015.
[5]MADRY A,MAKELOV A,SCHMIDT L,et al.Towards Deep Learning Models Resistant to Adversarial Attacks[J].arXiv:1706.06083,2017.
[6]LUONG T,PHAM H,MANNING C D.Effective Approaches to Attention-based Neural Machine Translation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.ACL,2015:1412-1421.
[7]KIM Y.Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP).ACL,2014:1746-1751.
[8]KREUTZ T,DAELEMANS W.Enhancing General SentimentLexicons for Domain-Specific Use [C]//Proceedings of the 27th International Conference on Computational Linguistics.ACL,2018:1056-1064.
[9]TENG Z Y,VO D T,ZHANG Y.Context-Sensitive LexiconFeatures for Neural Sentiment Analysis[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.ACL,2016:1629-1638.
[10]LIU Y H,OTT M,GOYAL N,et al.RoBERTa:A Robustly Optimized BERT Pretraining Approach[J].arXiv:1907.11692,2019.
[11]LAN Z Z,CHEN M D,GOODMAN S,et al.ALBERT:A Lite BERT for Self-supervised Learning of Language Representations[J].arXiv:1909.11942,2019.
[12]CLARK K,LUONG M T,LE Q V,et al.ELECTRA:Pre-training Text Encoders as Discriminators Rather Than Generators[J].arXiv:2003.10555,2020.
[13]JOSHI M,CHEN D Q,LIU Y H,et al.SpanBERT:Improving Pre-training by Representing and Predicting Spans[J].Transactions of the Association for Computational Linguistics,2020,8:64-77.
[14]SANH V,DEBUT L,CHAUMOND J,et al.DistilBERT,a distilled version of BERT:smaller,faster,cheaper and lighter[J].arXiv:1910.01108,2019.
[15]FENG Y,CHENG Y.Short Text Sentiment Analysis Based on Multi-Channel CNN With Multi-Head Attention Mechanism[J].IEEE Access,2021,9:19854-19863.
[16]PENG Z,LU Y,PAN S,et al.Efficient Speech Emotion Recognition Using Multi-Scale CNN and Attention[C]//IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2021).2021:3020-3024.
[17]YANG H,ZENG B Q,YANG J H,et al.A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction[J].arXiv:1912.07976,2019.
[18]ZHAO H T,MA C,DONG X S,et al.Certified Robustness Against Natural Language Attacks by Causal Intervention[C]//International Conference on Machine Learning.2022.
[19]XU Z E,ZHU G H,MENG C H,et al.A2:Efficient Automated Attacker for Boosting Adversarial Training[J].arXiv:2210.03543,2022.
[20]MA X S,WANG Z K,LIU W W.On the Tradeoff Between Robustness and Fairness[C]//Advances in Neural Information Processing Systems.2022.
[21]LI X Y,XIN Z,LIU W W.Defending Against Adversarial Attacks via Neural Dynamic System[C]//Advances in Neural Information Processing Systems.2022.
[22]MA F K,HU X M,LIU A W,et al.AMR-based Network for Aspect-based Sentiment Analysis[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics,2023:322-337.
[23]ZHANG H Y,DUAN L G,WANG Q C,et al.Multi-entity sentiment analysis of long text based on multi-task joint training[J].Computer Science,2024,51(6):309-316.
[24]LIU D X,DUAN L G,CUI J J,et al.Short text semantic ma-tching strategy based on dual channels of semantic similarity matrix and word vector[J].Computer Science,2024,51(12):250-258.
[25]WU Y H,SCHUSTER M,CHEN Z F,et al.Google's Neural Machine Translation System:Bridging the Gap between Human and Machine Translation[J].arXiv:1609.08144,2016.
[26]SUN Z J,LI X Y,SUN X F,et al.ChineseBERT:Chinese Pretraining Enhanced by Glyph and Pinyin Information[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing.ACL,2021:2065-2075.
[27]LI X L,LEI Y Y,JI S W.BERT- and BiLSTM-Based Sentiment Analysis of Online Chinese Buzzwords[J].Future Internet,2022,14(11):332.
[28]DAI J,CHEN C.Text classification system of academic papers based on hybrid Bert-BiGRU model[C]//2020 12th International Conference on Intelligent Human-Machine Systems and Cybernetics(IHMSC).2020:40-44.
[29]HASIBK M,AZAM S,KARIM A,et al.,MCNN-LSTM:Combining CNN and LSTM to Classify Multi-Class Text in Imba-lanced News Data[J].IEEE Access,2023,11:93048-93063.
[30]HE Y S.BERT-CNN-BiLSTM:A Hybrid Deep Learning Modelfor Accurate Sentiment Analysis[C]//IEEE 5th International Conference on Power,Intelligent Computing and Systems(ICPICS 2023).2023:921-926.
[1] WANG Baocai, WU Guowei. Interpretable Credit Risk Assessment Model:Rule Extraction Approach Based on AttentionMechanism [J]. Computer Science, 2025, 52(10): 50-59.
[2] ZHENG Hanyuan, GE Rongjun, HE Shengji, LI Nan. Direct PET to CT Attenuation Correction Algorithm Based on Imaging Slice Continuity [J]. Computer Science, 2025, 52(10): 115-122.
[3] XU Hengyu, CHEN Kun, XU Lin, SUN Mingzhai, LU Zhou. SAM-Retina:Arteriovenous Segmentation in Dual-modal Retinal Image Based on SAM [J]. Computer Science, 2025, 52(10): 123-133.
[4] WEN Jing, ZHANG Songsong, LI Xufeng. Target Tracking Method Based on Cross Scale Fusion of Features and Trajectory Prompts [J]. Computer Science, 2025, 52(10): 144-150.
[5] SHENG Xiaomeng, ZHAO Junli, WANG Guodong, WANG Yang. Immediate Generation Algorithm of High-fidelity Head Avatars Based on NeRF [J]. Computer Science, 2025, 52(10): 159-167.
[6] ZHENG Dichen, HE Jikai, LIU Yi, GAO Fan, ZHANG Dengyin. Low Light Image Adaptive Enhancement Algorithm Based on Retinex Theory [J]. Computer Science, 2025, 52(10): 168-175.
[7] RUAN Ning, LI Chun, MA Haoyue, JIA Yi, LI Tao. Review of Quantum-inspired Metaheuristic Algorithms and Its Applications [J]. Computer Science, 2025, 52(10): 190-200.
[8] XIONG Zhuozhi, GU Zhouhong, FENG Hongwei, XIAO Yanghua. Subject Knowledge Evaluation Method for Language Models Based on Multiple ChoiceQuestions [J]. Computer Science, 2025, 52(10): 201-207.
[9] WANG Jian, WANG Jingling, ZHANG Ge, WANG Zhangquan, GUO Shiyuan, YU Guiming. Multimodal Information Extraction Fusion Method Based on Dempster-Shafer Theory [J]. Computer Science, 2025, 52(10): 208-216.
[10] CHEN Yuyan, JIA Jiyuan, CHANG Jingwen, ZUO Kaiwen, XIAO Yanghua. SPEAKSMART:Evaluating Empathetic Persuasive Responses by Large Language Models [J]. Computer Science, 2025, 52(10): 217-230.
[11] LI Sihui, CAI Guoyong, JIANG Hang, WEN Yimin. Novel Discrete Diffusion Text Generation Model with Convex Loss Function [J]. Computer Science, 2025, 52(10): 231-238.
[12] ZHANG Jiawei, WANG Zhongqing, CHEN Jiali. Multi-grained Sentiment Analysis of Comments Based on Text Generation [J]. Computer Science, 2025, 52(10): 239-246.
[13] WANG Ye, WANG Zhongqing. Text Simplification for Aspect-based Sentiment Analysis Based on Large Language Model [J]. Computer Science, 2025, 52(10): 258-265.
[14] ZHAO Jinshuang, HUANG Degen. Summary Faithfulness Evaluation Based on Data Augmentation and Two-stage Training [J]. Computer Science, 2025, 52(10): 266-274.
[15] SUN Liangxu, LI Linlin, LIU Guoli. Sub-problem Effectiveness Guided Multi-objective Evolution Algorithm [J]. Computer Science, 2025, 52(10): 296-307.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!