Computer Science ›› 2019, Vol. 46 ›› Issue (6): 206-211.doi: 10.11896/j.issn.1002-137X.2019.06.031

Previous Articles     Next Articles

BLSTM_MLPCNN Model for Short Text Classification

ZHENG Cheng, HONG Tong-tong, XUE Man-yi   

  1. (School of Computer Science and Technology,Anhui University,Hefei 230601,China)
  • Received:2018-05-18 Published:2019-06-24

Abstract: Text representation and text feature extraction are essential procedures in natural language processing and directly affect text classification performance.The major output of the present work is the establishment of the BLSTM_MLPCNN neural network model whose inputs are character-level vector integrated with word vector.In this model,firstly the character-level vector is obtained from character via convolutional neural network (CNN),and is integrated with the word vector to compose the pre-training words embedded vectors (also an input to BLSTM model).Then the combination of the BLSTM model’s forward output,word embedded vector and backward output forms the document feature map,and finally the MLPCNN model is used to extract feature.The experiments on the pertinent datasets prove the classification performance of BLSTM_MLPCNN model is superior to CNN model,RNN model and CNN/RNN combinatorial model.

Key words: Bidirectional long short-term memory network(BLSTM), Character-level vector, Convolutional neural network(CNN), Multi-layer perceptron convolutional neural network(MLPCNN), Multi-layer perceptron(MLP), Word vector

CLC Number: 

  • TP391.1
[1]HINTON G E.Learning distributed representations of concepts[C]∥Proceedings of the Eighth Annual Conference of the Cognitive Science Society.1986:1-12.
[2]CAI H P.Research on short text classification based on convolutional neural network[D].Chongqing:Southwest University,2016.(in Chinese)
蔡慧苹.基于卷积神经网络的短文本分类方法研究[D].重庆:西南大学,2016.
[3]WANG F,WANG Z,LI Z,et al.Concept-based short text classification and ranking[C]∥Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management.ACM,2014:1069-1078.
[4]NING Y H,FAN X H,WU Y.Short text classification based on domain word ontology[J].Computer Science,2009,36(3):142-145.(in Chinese)
宁亚辉,樊兴华,吴渝.基于领域词语本体的短文本分类[J].计算机科学,2009,36(3):142-145.
[5]RAO G Q,YU D,XUN E D.Unsupervised text feature extraction based on natural annotation information and implicit topic model[J].Journal of Chinese Information Processing,2015,29(6):141-149.(in Chinese)
饶高琦,于东,荀恩东.基于自然标注信息和隐含主题模型的无监督文本特征抽取[J].中文信息学报,2015,29(6):141-149.
[6]SRIRAM B,FUHRY D,DEMIR E,et al.Short text classification in Twitter to improve information filtering[C]∥Procee-dings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York:ACM,2010:841-842.
[7]GODIN F,SLAVKOVIKJ V,DE N W,et al.Using topic models for Twitter hashtag recommendation[C]∥Proceedings of the 22nd International Conference on World Wide Web.New York:ACM,2013:593-596.
[8]MEHROTRA R,SANNER S,BUNTINE W,et al.Improving LDA topic models for Microblogs via Tweet pooling and automatic labeling[C]∥Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval.New York:ACM,2013:889-892.
[9]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P A.Convolutional neural network for modelling sentences[J].ar-Xiv:1404.2188,2014.
[10]LEI T,BARZILAY R,JAAKKOLA T.Molding CNNs for text:non-linear,non-consecutive convolutions[J].Indiana University Mathematics Journal,2015,58(3):1151-1186.
[11]WANG P.Semantic clustering and convolutional neural network for short text categorization[C]∥Proceeding of Meeting of the Association for Computional Linguistics.ACL,2015:352-357.
[12]JOULIN A,GRAVE E,BOJANOWSKI P,et al.Bag of tricks for efficient text classification[C]∥Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics.ACL,2017:427-431.
[13]AREVIAN G.Recurrent neural networks for robust real-world text classification[C]∥IEEE/WIC/ACM International Confe-rence on Web Intelligence.Washington,DC:IEEE,2007:326-329.
[14]TANG D Y,QIN B,LIU T.Document modeling with gated recurrent neural network for sentiment classification[C]∥Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.EMNL,2015:1422-1432.
[15]ZHOU C H,SUN C,LIU Z,et al.A C-LSTM neural network for text classification[J].Computer Science,2015,1(4):39-44.
[16]LAI S W,XU L H,LIU K,et al.Recurrent convolutional neural networks for text classification[C]∥Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence.AAAI,2016:2268-2273.
[1] JIANG Sheng-teng, ZHANG Yi-chi, LUO Peng, LIU Yue-ling, CAO Kuo, ZHAO Hai-tao, WEI Ji-bo. Analysis of Performance Metrics of Semantic Communication Systems [J]. Computer Science, 2022, 49(7): 236-241.
[2] LIU Shuo, WANG Geng-run, PENG Jian-hua, LI Ke. Chinese Short Text Classification Algorithm Based on Hybrid Features of Characters and Words [J]. Computer Science, 2022, 49(4): 282-287.
[3] YANG Jin-cai, CAO Yuan, HU Quan, SHEN Xian-jun. Relation Classification of Chinese Causal Compound Sentences Based on Transformer Model and Relational Word Feature [J]. Computer Science, 2021, 48(6A): 295-298.
[4] YANG Qing, ZHANG Ya-wen, ZHU Li, WU Tao. Text Sentiment Analysis Based on Fusion of Attention Mechanism and BiGRU [J]. Computer Science, 2021, 48(11): 307-311.
[5] JING Li, LI Man-man, HE Ting-ting. Sentiment Classification of Network Reviews Combining Extended Dictionary and Self-supervised Learning [J]. Computer Science, 2020, 47(11A): 78-82.
[6] LI Zhou-jun,WANG Chang-bao. Survey on Deep-learning-based Machine Reading Comprehension [J]. Computer Science, 2019, 46(7): 7-12.
[7] WANG Le-le,WANG Bin-qiang,LIU Jian-gang,ZHANG Jian-hui,MIAO Qi-guang. Study on Malicious Program Detection Based on Recurrent Neural Network [J]. Computer Science, 2019, 46(7): 86-90.
[8] SHAN Yi-dong, WANG Heng-jun, HUANG He, YAN Qian. Study on Named Entity Recognition Model Based on Attention Mechanism——Taking Military Text as Example [J]. Computer Science, 2019, 46(6A): 111-114.
[9] ZHANG Ming-yue, WANG Jing. Interactive Likelihood Target Tracking Algorithm Based on Deep Learning [J]. Computer Science, 2019, 46(2): 279-285.
[10] WU Chen, YUAN Yu-wei, WANG Hong-wei, LIU Yu, LIU Si-tong, QUAN Ji-cheng. Word Vectors Fusion Based Remote Sensing Scenes Zero-shot Classification Algorithm [J]. Computer Science, 2019, 46(12): 286-291.
[11] ZHENG Cheng, XUE Man-yi, HONG Tong-tong, SONG Fei-bao. DC-BiGRU_CNN Model for Short-text Classification [J]. Computer Science, 2019, 46(11): 186-192.
[12] SHI Kai-jing, BAO Hong. Forward Vehicle Detection Research Based on Improved FAST R-CNN Network [J]. Computer Science, 2018, 45(6A): 179-182.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!