计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (10): 247-257.doi: 10.11896/jsjkx.240800061

• 人工智能 • 上一篇    下一篇

基于大批次对抗策略和强化特征提取的文本情感分类方法

陈嘉昊1, 段利国1,2, 常轩伟1, 李爱萍1, 崔娟娟1, 郝渊斌1   

  1. 1 太原理工大学计算机科学与技术学院 太原 030024
    2 山西电子科技学院 山西 临汾 041000
  • 收稿日期:2024-08-12 修回日期:2024-11-30 出版日期:2025-10-15 发布日期:2025-10-14
  • 通讯作者: 段利国(zhaixing202202@163.com)
  • 作者简介:(1931668813@qq.com)
  • 基金资助:
    山西省自然科学基金(202203021221234,202303021211052)

Text Sentiment Classification Method Based on Large-batch Adversarial Strategy and EnhancedFeature Extraction

CHEN Jiahao1, DUAN Liguo1,2, CHANG Xuanwei1, LI Aiping1, CUI Juanjuan1, HAO Yuanbin1   

  1. 1 College of Computer Science and Technology,Taiyuan University of Technology,Taiyuan 030024,China
    2 Shanxi Electronic Science and Technology Institute,Linfen,Shanxi 041000,China
  • Received:2024-08-12 Revised:2024-11-30 Online:2025-10-15 Published:2025-10-14
  • About author:DUAN Liguo,born in 1970,is a member of CCF(No.15823S).His main research interest is natural language processing.
  • Supported by:
    Natural Science Foundation of Shanxi Province,China(202203021221234,202303021211052).

PDF (PC)

0

摘要： 文本情感分类任务旨在对短文本语句进行分析并判断其对应的情感类别。为解决现有模型在情感分类方面缺乏大规模高质量语料数据集、文本特征非均匀重要性提取不足等问题,提出了一种基于大批次对抗策略和强化特征提取的文本情感分类方法。首先将文本数据集输入预训练语言模型BERT中,得到相应的词嵌入向量表示;再利用BiLSTM进一步学习序列中的上下文依赖关系;之后将局部注意力机制与TextCNN的局部感受野加权结合,实现强化特征提取能力;最后将BiLSTM的输出与TextCNN的输出进行拼接,得到两个空间的深层特征融合,再交由分类器进行情感分类的判断。整个训练过程采取大批次对抗策略,在词嵌入空间中加入对抗性扰动并进行多次迭代,进而提高模型的鲁棒性。在多个数据集上的实验结果验证了该模型的有效性。

关键词: 短文本, 情感分类, 对抗策略, 特征提取, 词嵌入

Abstract: The text sentiment classification task aims to analyze short text sentences and determine their corresponding sentiment categories.In order to solve the problems of lack of large-scale high-quality corpus dataset and insufficient non-uniform importance extraction of text features in the existing models in sentiment classification,this paper proposes a text sentiment classification method based on large-batch adversarial strategy and enhanced feature extraction.Firstly,the text dataset is input into the pre-trained language model BERT to obtain the corresponding word embedding vector representation,and then the BiLSTM is used to further learn the context dependencies in the sequence.Then,the local attention mechanism is combined with the local receptive field weighting of TextCNN to enhance the feature extraction ability.Finally,the output of BiLSTM and the output of TextCNN are spliced to obtain the deep feature fusion of the two spaces,which are handed over to the classifier for the judgment of sentiment classification.In the whole training process,a large-batch adversarial strategy is adopted,and adversarial perturbations are added to the word embedding space and multiple iterations are carried out to improve the robustness of the model.Experimental results on multiple datasets verify the effectiveness of the proposed model.

Key words: Short text,Sentiment classification,Adversarial strategy,Feature extraction,Word embeddings

中图分类号: 

  • TP391
[1]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.ACL,2019:4171-4186.
[2]ZHAO W M,ALWIDIAN S A,MAHMOUD Q H.Adversarial Training Methods for Deep Learning:A Systematic Review[J].Algorithms,2022,15(8):283.
[3]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[4]HUANG Z H,XU W,YU K.Bidirectional LSTM-CRF Models for Sequence Tagging[J].arXiv:1508.01991,2015.
[5]MADRY A,MAKELOV A,SCHMIDT L,et al.Towards Deep Learning Models Resistant to Adversarial Attacks[J].arXiv:1706.06083,2017.
[6]LUONG T,PHAM H,MANNING C D.Effective Approaches to Attention-based Neural Machine Translation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.ACL,2015:1412-1421.
[7]KIM Y.Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP).ACL,2014:1746-1751.
[8]KREUTZ T,DAELEMANS W.Enhancing General SentimentLexicons for Domain-Specific Use [C]//Proceedings of the 27th International Conference on Computational Linguistics.ACL,2018:1056-1064.
[9]TENG Z Y,VO D T,ZHANG Y.Context-Sensitive LexiconFeatures for Neural Sentiment Analysis[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.ACL,2016:1629-1638.
[10]LIU Y H,OTT M,GOYAL N,et al.RoBERTa:A Robustly Optimized BERT Pretraining Approach[J].arXiv:1907.11692,2019.
[11]LAN Z Z,CHEN M D,GOODMAN S,et al.ALBERT:A Lite BERT for Self-supervised Learning of Language Representations[J].arXiv:1909.11942,2019.
[12]CLARK K,LUONG M T,LE Q V,et al.ELECTRA:Pre-training Text Encoders as Discriminators Rather Than Generators[J].arXiv:2003.10555,2020.
[13]JOSHI M,CHEN D Q,LIU Y H,et al.SpanBERT:Improving Pre-training by Representing and Predicting Spans[J].Transactions of the Association for Computational Linguistics,2020,8:64-77.
[14]SANH V,DEBUT L,CHAUMOND J,et al.DistilBERT,a distilled version of BERT:smaller,faster,cheaper and lighter[J].arXiv:1910.01108,2019.
[15]FENG Y,CHENG Y.Short Text Sentiment Analysis Based on Multi-Channel CNN With Multi-Head Attention Mechanism[J].IEEE Access,2021,9:19854-19863.
[16]PENG Z,LU Y,PAN S,et al.Efficient Speech Emotion Recognition Using Multi-Scale CNN and Attention[C]//IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2021).2021:3020-3024.
[17]YANG H,ZENG B Q,YANG J H,et al.A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction[J].arXiv:1912.07976,2019.
[18]ZHAO H T,MA C,DONG X S,et al.Certified Robustness Against Natural Language Attacks by Causal Intervention[C]//International Conference on Machine Learning.2022.
[19]XU Z E,ZHU G H,MENG C H,et al.A2:Efficient Automated Attacker for Boosting Adversarial Training[J].arXiv:2210.03543,2022.
[20]MA X S,WANG Z K,LIU W W.On the Tradeoff Between Robustness and Fairness[C]//Advances in Neural Information Processing Systems.2022.
[21]LI X Y,XIN Z,LIU W W.Defending Against Adversarial Attacks via Neural Dynamic System[C]//Advances in Neural Information Processing Systems.2022.
[22]MA F K,HU X M,LIU A W,et al.AMR-based Network for Aspect-based Sentiment Analysis[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics,2023:322-337.
[23]ZHANG H Y,DUAN L G,WANG Q C,et al.Multi-entity sentiment analysis of long text based on multi-task joint training[J].Computer Science,2024,51(6):309-316.
[24]LIU D X,DUAN L G,CUI J J,et al.Short text semantic ma-tching strategy based on dual channels of semantic similarity matrix and word vector[J].Computer Science,2024,51(12):250-258.
[25]WU Y H,SCHUSTER M,CHEN Z F,et al.Google's Neural Machine Translation System:Bridging the Gap between Human and Machine Translation[J].arXiv:1609.08144,2016.
[26]SUN Z J,LI X Y,SUN X F,et al.ChineseBERT:Chinese Pretraining Enhanced by Glyph and Pinyin Information[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing.ACL,2021:2065-2075.
[27]LI X L,LEI Y Y,JI S W.BERT- and BiLSTM-Based Sentiment Analysis of Online Chinese Buzzwords[J].Future Internet,2022,14(11):332.
[28]DAI J,CHEN C.Text classification system of academic papers based on hybrid Bert-BiGRU model[C]//2020 12th International Conference on Intelligent Human-Machine Systems and Cybernetics(IHMSC).2020:40-44.
[29]HASIBK M,AZAM S,KARIM A,et al.,MCNN-LSTM:Combining CNN and LSTM to Classify Multi-Class Text in Imba-lanced News Data[J].IEEE Access,2023,11:93048-93063.
[30]HE Y S.BERT-CNN-BiLSTM:A Hybrid Deep Learning Modelfor Accurate Sentiment Analysis[C]//IEEE 5th International Conference on Power,Intelligent Computing and Systems(ICPICS 2023).2023:921-926.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发