Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (4): 288-293.doi: 10.11896/jsjkx.211100016

• Artificial Intelligence • Previous Articles     Next Articles

Text Classification Method Based on Word2Vec and AlexNet-2 with Improved AttentionMechanism

ZHONG Gui-feng1, PANG Xiong-wen2, SUI Dong3   

  1. 1 College of Computer Science&Engineering, Guangzhou Institute of Science and Technology, Guangzhou 510540, China;
    2 School of Computer, South China Normal University, Guangzhou 530631, China;
    3 School of Electrical and Information Engineering, Beijing University of Civil Engineering and Architecture, Beijing 102406, China
  • Received:2021-11-01 Revised:2022-01-23 Published:2022-04-01
  • About author:ZHONG Gui-feng,born in 1983,postgraduate,lecturer.Her main research interests include data analysis and mi-ning,machine learning and applications of artificial intelligence.
  • Supported by:
    This work was supported by the National Natural Science Youth Fund(61702026),2020 Guangdong University Scientific Research Project(2020GXJK201),2019 Guangdong University Scientific Research Project(2019KTSCX243) and Special Program for Higher Education in Guangdong Province in 2021(2021GXJK275).

PDF (PC)

661

Abstract: In order to improve the accuracy and efficiency of text classification, a text classification method based on Word2Vec text representation and AlexNet-2 with improved attention mechanism is proposed.Firstly, Word2Vec is adopted to embed the text word features, and the word vector is trained to represent the text in the form of distributed vectors.Then, an improved AlexNet-2 is used to effectively encode the long-distance word dependency.Meanwhile, the attention mechanism is added to the model to learn the contextual embedding semantics of the target word efficiently, and the word weight is adjusted according to the correlation between the input of word vector and the final prediction result.The experiment is evaluated in three public data sets, and the situations of a large number of sample annotations and a small number of sample annotations are analyzed.Experimental results show that, compared with the existing excellent methods, the proposed method can significantly improve the performance and efficiency of text classification.

Key words: AlexNet-2 model, Attention mechanism, Contextual embedding, Text classification, Word dependency

CLC Number: 

  • TP391
[1] LUO X.Efficient English Text Classification Using SelectedMachine Learning Techniques[J].Alexandria Engineering Journal,2021,60(3):3401-3409.
[2] AL-SALEMI B,AYOB M,NOAH S.Feature Ranking for Enhancing Boosting-based Multi-label Text Categorization[J].Expert Systems with Applications,2018,113(12):531-543.
[3] JIN G,XU L.Long Text Classification Method,Device,Com-puter Equipment and Storage Medium Based on Word Bag Mo-del:CN Patent 110096591 A[P].2019.
[4] XU A D,ZHAO Y K,ZHANG Y Q,et al.Text Classification Method Based on Heterogeneous Space and Multiple-Classifier Fusion[J].Journal of Sichuan Ordnance,2019,40(12):136-141.
[5] AL-SALEMI B,AB-AZIZ M J,MOHD-NOAH S A.et al.LDA-AdaBoost.MH:Accelerated AdaBoost.MH Based on Latent Dirichlet Allocation for Text Categorization[J].Journal of Information Science,2015,41(1):27-40.
[6] SHI Z S,DU Y,DU T,et al.The Turnout Abnormality Diagnosis Based on Semi-Supervised Learning Method[J].International Journal of Software Engineering and Knowledge Engineering,2020,30(7):961-976.
[7] WU Y J,LI J,SONG C F,et al.High Utility Neural Networks for Text Classification[J].Acta Electronica Sinica,2020,48(2):279-284.
[8] DU L,CAO D,LIN S Y,et al.Extraction and Automatic Classi-fication of TCM Medical Records Based on Attention Mechanism of BERT and Bi-LSTM[J].Computer Science,2020,47(S2):426-430.
[9] YAO L,MAO C S,LUO Y.Graph Convolutional Networks for Text Classification[J].arXiv:1809.05679,2018.
[10] RAGESH R,SELLAMANNICKAM S,IYER A,et al.HeteGCN:Heterogeneous Graph Convolutional Networks for Text Classification[C]//The Fourteenth ACM International Conference on Web Search and Data Mining.Virtual Event,Israel:ACM,2021:105-115.
[11] KIPF T N,WELLING M.Semi-Supervised Classification withGraph Convolutional Networks[C]//International Conference on Learning Representations.Toulon,France:IEEE,2017:109-117.
[12] XIAO L,CHEN B L,HUANG X,et al.Multi-label Text Classification Method Based on Label Semantic Information[J].Journal of Software,2020,31(4):1079-1089.
[13] JANG B,KIM M,HARERIMANA G,et al.Bi-LSTM Model to Increase Accuracy in Text Classification:Combining Word2vec CNN and Attention Mechanism[J].Applied Sciences,2020,10(17):5841-5750.
[14] LUO X,WANG X H.Research on Multi-feature Fusion Text Classification Model Based on Self-attention Mechanism[J].Journal of Physics:Conference Series,2020,1693(1):012071-012077.
[15] TU N,THU H,NGUYEN V A.Language Model Combinedwith Word2Vec for Product’s Aspect Based Extraction[J].ICIC Express Letters,2020,14(11):1033-1040.
[16] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet Classification with Deep Convolutional Neural Networks[J].Advances in Neural Information Processing Systems,2012,25(2):1-9.
[17] KRIZHEVSKY A,SUTSKEVER I,GEOFFREY E.ImageNet Classification with Deep Convolutional Neural Networks[J].Communications of the ACM,2017,60(6):84-90.
[18] BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[J].Computer Science,2014,47(3):1-15.
[19] LIU J Y,JIA B B.Combining One-vs-One Decomposition andInstance-Based Learning for Multi-Class Classification[J].IEEE Access,2020,85(8):499-507.
[20] GAO C L,XU H,GAO K.Attention-based Bi-LSTM Network with Part-of-speech Features for Chinese Text Classification[J].Journal of Hebei University of Science and Technology,2018,39(5):73-80.
[21] WANG T,LI M.Research on Comment Text Mining Based on LDA Model and Semantic Network[J].Journal of Chongqing Technology and Business University(Natural Science Edition),2019,36(4):9-16.
[1] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[3] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[4] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[5] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[6] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] WU Hong-xin, HAN Meng, CHEN Zhi-qiang, ZHANG Xi-long, LI Mu-hang. Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning [J]. Computer Science, 2022, 49(8): 12-25.
[9] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[10] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[11] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[12] TAN Ying-ying, WANG Jun-li, ZHANG Chao-bo. Review of Text Classification Methods Based on Graph Convolutional Network [J]. Computer Science, 2022, 49(8): 205-216.
[13] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[14] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[15] XIONG Luo-geng, ZHENG Shang, ZOU Hai-tao, YU Hua-long, GAO Shang. Software Self-admitted Technical Debt Identification with Bidirectional Gate Recurrent Unit and Attention Mechanism [J]. Computer Science, 2022, 49(7): 212-219.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.