Computer Science ›› 2020, Vol. 47 ›› Issue (6A): 12-16.doi: 10.11896/JsJkx.200200076

• Artificial Intelligence • Previous Articles     Next Articles

Text Representation and Classification Algorithm Based on Adversarial Training

ZHANG Xiao-hui1, YU Shuang-yuan1, WANG Quan-xin2 and XU Bao-min1   

  1. 1 School of Computer and Information Technology,BeiJing Jiaotong University,BeiJing 100044,China
    2 School of Computer and Information Technology,BeiJing Jiaotong University Haibin College,Huanghua,Hebei 061199,China
  • Published:2020-07-07
  • About author:ZHANG Xiao-hui, born in 1995, postgraduate, is a member of China Computer Federation.Her main research interest includes natural language processing.
    WANG Quan-xin, born in 1984, master’sdegree, lecturer.Her main research interests include Java and database.
  • Supported by:
    This work was supported by the Key ProJects of Science and Technology Research of Hebei Province Higher Education(ZD2017304).

Abstract: Text representation and classification are hot topics in the field of natural language understanding.There are many text classification methods,including convolutional networks,recursive networks,self-attention mechanisms and their combinations.complex networks cannot fundamentally improve the performance of classificationtext representation is the key to text classification.In order to obtain a good text representation and improve the performance of text classification,an LSTM-based representation learning-text classification model is constructed,where the representation learning model uses a language model to provide the text classification model with initialized text representation and network parameters.main work is to adversarial training methods that is,add perturbations to word vectors to construct adversarial samples,and train the original samples.By improving the model’s ability adversarial samples,the quality of text representation,and the generalization performance of the model, the classification effect of the classification model.xperimental results show that the method based on adversarial training achieves 92.9%,93.2% and 98.9% on the benchmark datasets AGNews,IMDBDBpedia,that the method can improve the classification effect of the model.

Key words: Adversarial training, Text classification, Text representation

CLC Number: 

  • TP391
[1] PETERS M E,NEUMANN M,IYYER M,et al.Deep contextualized word representations.arXiv:18020.05365,2018.
[2] RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving language understanding by generative pre-training.https://s3-us-west-2.amazonaws.com/openai-assets/researchcovers/languageunsupervised/language understanding paper.pdf.
[3] DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding.arXiv:1810.04805,2018.
[4] GOODFELLOW I J,SHLENS J,SZEGEDY C.Explaining and harnessing adversarial examples.arXiv:1412.6572,2014.
[5] MIYATO T,DAI A M,GOODFELLOW I.Adversarial training methods for semi-supervised text classification.arXiv:1605.07725,2016.
[6] WANG D,GONG C,LIU Q.Improving Neural Language Mo-deling via Adversarial Training//International Conference on Machine Learning.2019.
[7] BENGIO Y,DUCHARME R,VINCENT P,et al.Aneural probabilistic language model.Journal of Machine Learning Research,2003,3(Feb):1137-1155.
[8] KIM Y.Convolutional neural networks for sentence classification.arXiv:1408.5882,2014.
[9] SZEGEDY C,ZAREMBA W,SUTSKEVER I,et al.Intriguing properties of neural networks.arXiv:1312.6199,2013.
[10] DAI A M,LE Q V.Semi-supervised sequence learning//Advances in Neural Information Processing Systems.2015:3079-3087.
[11] QIAO C,HUANG B,NIU G,et al.A New Method of Region Embedding for Text Classification//ICLR.2018.
[12] LETARTE G,PARADIS F,GIGURE P,et al.Importance of self-attention for sentiment analysis//Proceedings of the 2018 EMNLP Workshop BlackboxNLP:Analyzing and Interpreting Neural Networks for NLP.2018:267-275.
[13] WANG B.Disconnected recurrent neural networks for text categorization//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2018:2311-2320.
[14] NIU G,XU H,HE B,et al.Enhancing Local Feature Extraction with Global Representation for Neural Text Classification//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).2019:496-506.
[15] JOULIN A,GRAVE E,BOJANOWSKI P,et al.Bag of tricks for efficient text classification.arXiv:1607.01759,2016.
[16] SHEN D,WANG G,WANG W,et al.Baseline needs more love:On simple word-embedding-based models and associated pooling mechanisms.arXiv:1805.09843,2018.
[17] CHEN W,SU Y,SHEN Y,et al.How large avocabulary does text classification need? a variational approach to vocabulary selection.arXiv:1902.10339,2019.
[1] TAN Ying-ying, WANG Jun-li, ZHANG Chao-bo. Review of Text Classification Methods Based on Graph Convolutional Network [J]. Computer Science, 2022, 49(8): 205-216.
[2] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[3] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[4] WU Hong-xin, HAN Meng, CHEN Zhi-qiang, ZHANG Xi-long, LI Mu-hang. Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning [J]. Computer Science, 2022, 49(8): 12-25.
[5] YAN Meng, LIN Ying, NIE Zhi-shen, CAO Yi-fan, PI Huan, ZHANG Lan. Training Method to Improve Robustness of Federated Learning [J]. Computer Science, 2022, 49(6A): 496-501.
[6] SHAO Xin-xin. TI-FastText Automatic Goods Classification Algorithm [J]. Computer Science, 2022, 49(6A): 206-210.
[7] DENG Kai, YANG Pin, LI Yi-zhou, YANG Xing, ZENG Fan-rui, ZHANG Zhen-yu. Fast and Transmissible Domain Knowledge Graph Construction Method [J]. Computer Science, 2022, 49(6A): 100-108.
[8] KANG Yan, WU Zhi-wei, KOU Yong-qi, ZHANG Lan, XIE Si-yu, LI Hao. Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution [J]. Computer Science, 2022, 49(6A): 150-158.
[9] XU Guo-ning, CHEN Yi-peng, CHEN Yi-ming, CHEN Jin-yin, WEN Hao. Data Debiasing Method Based on Constrained Optimized Generative Adversarial Networks [J]. Computer Science, 2022, 49(6A): 184-190.
[10] DENG Zhao-yang, ZHONG Guo-qiang, WANG Dong. Text Classification Based on Attention Gated Graph Neural Network [J]. Computer Science, 2022, 49(6): 326-334.
[11] LIU Shuo, WANG Geng-run, PENG Jian-hua, LI Ke. Chinese Short Text Classification Algorithm Based on Hybrid Features of Characters and Words [J]. Computer Science, 2022, 49(4): 282-287.
[12] ZHONG Gui-feng, PANG Xiong-wen, SUI Dong. Text Classification Method Based on Word2Vec and AlexNet-2 with Improved AttentionMechanism [J]. Computer Science, 2022, 49(4): 288-293.
[13] DENG Wei-bin, ZHU Kun, LI Yun-bo, HU Feng. FMNN:Text Classification Model Fused with Multiple Neural Networks [J]. Computer Science, 2022, 49(3): 281-287.
[14] ZHANG Hu, BAI Ping. Graph Convolutional Networks with Long-distance Words Dependency in Sentences for Short Text Classification [J]. Computer Science, 2022, 49(2): 279-284.
[15] YANG Yang, CHEN Wei, ZHANG Dan-yi, WANG Dan-ni, SONG Shuang. Adversarial Attacks Threatened Network Traffic Classification Based on CNN [J]. Computer Science, 2021, 48(7): 55-61.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!