计算机科学 ›› 2020, Vol. 47 ›› Issue (6A): 12-16.doi: 10.11896/JsJkx.200200076
张晓辉1, 于双元1, 王全新2, 徐保民1
ZHANG Xiao-hui1, YU Shuang-yuan1, WANG Quan-xin2 and XU Bao-min1
摘要： 文本表示和分类是自然语言理解领域的研究热点。目前已有很多文本分类方法,包括卷积网络、递归网络、自注意力机制以及它们的结合。但是,复杂的网络并不能从根本上提高文本分类的性能,好的文本表示才是文本分类的关键。为了获得好的文本表示,提高文本分类性能,构建了基于LSTM的表示学习-文本分类模型,其中表示学习模型利用语言模型为文本分类模型提供初始化的文本表示和网络参数。文中主要采用对抗训练方法训练语言模型,即在词向量上添加扰动构造对抗样本,再利用对抗样本和原始样本一起训练模型,通过提升模型对对抗样本的正确分类能力,提高文本表示的质量,增强模型的泛化性能,从而改善分类模型的分类效果。实验结果表明,基于对抗训练的文本分类方法在基准数据集AGNews,IMDB,DBpedia上分别实现了92.9%,93.2%,98.9%的准确率,证明了该方法能够有效提高文本分类模型的分类性能。
中图分类号:
| [1] PETERS M E,NEUMANN M,IYYER M,et al.Deep contextualized word representations.arXiv:18020.05365,2018. [2] RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving language understanding by generative pre-training.https://s3-us-west-2.amazonaws.com/openai-assets/researchcovers/languageunsupervised/language understanding paper.pdf. [3] DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding.arXiv:1810.04805,2018. [4] GOODFELLOW I J,SHLENS J,SZEGEDY C.Explaining and harnessing adversarial examples.arXiv:1412.6572,2014. [5] MIYATO T,DAI A M,GOODFELLOW I.Adversarial training methods for semi-supervised text classification.arXiv:1605.07725,2016. [6] WANG D,GONG C,LIU Q.Improving Neural Language Mo-deling via Adversarial Training//International Conference on Machine Learning.2019. [7] BENGIO Y,DUCHARME R,VINCENT P,et al.Aneural probabilistic language model.Journal of Machine Learning Research,2003,3(Feb):1137-1155. [8] KIM Y.Convolutional neural networks for sentence classification.arXiv:1408.5882,2014. [9] SZEGEDY C,ZAREMBA W,SUTSKEVER I,et al.Intriguing properties of neural networks.arXiv:1312.6199,2013. [10] DAI A M,LE Q V.Semi-supervised sequence learning//Advances in Neural Information Processing Systems.2015:3079-3087. [11] QIAO C,HUANG B,NIU G,et al.A New Method of Region Embedding for Text Classification//ICLR.2018. [12] LETARTE G,PARADIS F,GIGURE P,et al.Importance of self-attention for sentiment analysis//Proceedings of the 2018 EMNLP Workshop BlackboxNLP:Analyzing and Interpreting Neural Networks for NLP.2018:267-275. [13] WANG B.Disconnected recurrent neural networks for text categorization//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2018:2311-2320. [14] NIU G,XU H,HE B,et al.Enhancing Local Feature Extraction with Global Representation for Neural Text Classification//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).2019:496-506. [15] JOULIN A,GRAVE E,BOJANOWSKI P,et al.Bag of tricks for efficient text classification.arXiv:1607.01759,2016. [16] SHEN D,WANG G,WANG W,et al.Baseline needs more love:On simple word-embedding-based models and associated pooling mechanisms.arXiv:1805.09843,2018. [17] CHEN W,SU Y,SHEN Y,et al.How large avocabulary does text classification need? a variational approach to vocabulary selection.arXiv:1902.10339,2019. |
| [1] | 檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064 |
| [2] | 闫佳丹, 贾彩燕. 基于双图神经网络信息融合的文本分类方法 Text Classification Method Based on Information Fusion of Dual-graph Neural Network 计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042 |
| [3] | 武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航. 监督和半监督学习下的多标签分类综述 Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning 计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111 |
| [4] | 郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077 |
| [5] | 邵欣欣. TI-FastText自动商品分类算法 TI-FastText Automatic Goods Classification Algorithm 计算机科学, 2022, 49(6A): 206-210. https://doi.org/10.11896/jsjkx.210500089 |
| [6] | 闫萌, 林英, 聂志深, 曹一凡, 皮欢, 张兰. 一种提高联邦学习模型鲁棒性的训练方法 Training Method to Improve Robustness of Federated Learning 计算机科学, 2022, 49(6A): 496-501. https://doi.org/10.11896/jsjkx.210400298 |
| [7] | 邓凯, 杨频, 李益洲, 杨星, 曾凡瑞, 张振毓. 一种可快速迁移的领域知识图谱构建方法 Fast and Transmissible Domain Knowledge Graph Construction Method 计算机科学, 2022, 49(6A): 100-108. https://doi.org/10.11896/jsjkx.210900018 |
| [8] | 康雁, 吴志伟, 寇勇奇, 张兰, 谢思宇, 李浩. 融合Bert和图卷积的深度集成学习软件需求分类 Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution 计算机科学, 2022, 49(6A): 150-158. https://doi.org/10.11896/jsjkx.210500065 |
| [9] | 徐国宁, 陈奕芃, 陈一鸣, 陈晋音, 温浩. 基于约束优化生成式对抗网络的数据去偏方法 Data Debiasing Method Based on Constrained Optimized Generative Adversarial Networks 计算机科学, 2022, 49(6A): 184-190. https://doi.org/10.11896/jsjkx.210400234 |
| [10] | 邓朝阳, 仲国强, 王栋. 基于注意力门控图神经网络的文本分类 Text Classification Based on Attention Gated Graph Neural Network 计算机科学, 2022, 49(6): 326-334. https://doi.org/10.11896/jsjkx.210400218 |
| [11] | 刘硕, 王庚润, 彭建华, 李柯. 基于混合字词特征的中文短文本分类算法 Chinese Short Text Classification Algorithm Based on Hybrid Features of Characters and Words 计算机科学, 2022, 49(4): 282-287. https://doi.org/10.11896/jsjkx.210200027 |
| [12] | 钟桂凤, 庞雄文, 隋栋. 基于Word2Vec和改进注意力机制AlexNet-2的文本分类方法 Text Classification Method Based on Word2Vec and AlexNet-2 with Improved AttentionMechanism 计算机科学, 2022, 49(4): 288-293. https://doi.org/10.11896/jsjkx.211100016 |
| [13] | 邓维斌, 朱坤, 李云波, 胡峰. FMNN:融合多神经网络的文本分类模型 FMNN:Text Classification Model Fused with Multiple Neural Networks 计算机科学, 2022, 49(3): 281-287. https://doi.org/10.11896/jsjkx.210200090 |
| [14] | 张虎, 柏萍. 融入句子中远距离词语依赖的图卷积短文本分类方法 Graph Convolutional Networks with Long-distance Words Dependency in Sentences for Short Text Classification 计算机科学, 2022, 49(2): 279-284. https://doi.org/10.11896/jsjkx.201200062 |
| [15] | 羊洋, 陈伟, 张丹懿, 王丹妮, 宋爽. 对抗攻击威胁基于卷积神经网络的网络流量分类 Adversarial Attacks Threatened Network Traffic Classification Based on CNN 计算机科学, 2021, 48(7): 55-61. https://doi.org/10.11896/jsjkx.210100095 |
|
||
首页