Computer Science ›› 2022, Vol. 49 ›› Issue (2): 279-284.doi: 10.11896/jsjkx.201200062

Special Issue: Natural Language Processing

• Artificial Intelligence • Previous Articles     Next Articles

Graph Convolutional Networks with Long-distance Words Dependency in Sentences for Short Text Classification

ZHANG Hu, BAI Ping   

  1. School of Computer and Information Technology,Shanxi University,Taiyuan 030006,China
  • Received:2020-12-07 Revised:2021-05-08 Online:2022-02-15 Published:2022-02-23
  • About author:ZHANG Hu,born in 1979,Ph.D,asso-ciate professor,is a member of China Computer Federation.His main research interests include natural language processing and representation learning.
  • Supported by:
    National Key Research and Development Program of China(2018YFB1005103),National Natural Science Foundation of China(62176145) and Natural Science Foundation of Shanxi Province,China(201901D111028).

Abstract: With the wide application of graph neural network technology in the field of natural language processing,the research of text classification based on graph neural networks has received more and more attention.Building graph for text is an important research task in the application of graph neural networks for text classification.Existing methods cannot effectively capture the dependency of long-distance words in sentences when building graph.Short text classification is a special type of text classification task in which the classified text is generally short,so the traditional text representation is usually sparse and lacks rich semantic information.Based on this,in this paper we propose a short text classification method based on graph convolutional neural networks incorporating long-distance words dependency.Firstly,by using the co-occurrence relationship of words,the containment relationship between documents and words,and the long-distance words dependency in sentences,a text graph is constructed for the entire text corpus.Then,the text graph is input into the graph convolutional neural networks,and the category label prediction is made for each document node after 2-layer convolution.The experimental results on the three datasets of online_shopping_10_cats,summaries of Chinese papers and hotel reviews show that the proposed method achieves better results than the existing baselines.

Key words: Building graph for text, Graph convolutional neural network, Natural language processing, Short text classification, Syntactic relationship

CLC Number: 

  • TP391
[1]KIM Y.Convolutional Neural Networks for Sentence Classifica-tion[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).2014:1746-1751.
[2]ZHOU C,SUN C,LIU Z,et al.A C-LSTM Neural Network forText Classification[J].Computerence,2015,1(4):39-44.
[3]WANG Y,HUANG M,ZHU X,et al.Attention-based LSTMfor Aspect-level Sentiment Classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:606-615.
[4]ZHOU J,CUI G,ZHANG Z,et al.Graph neural networks:Areview of methods and applications[J].arXiv:1812.08434,2018.
[5]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[6]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]//Advances in Neural Information Processing Systems.2017:1024-1034.
[7]CHEN J,MA T,XIAO C.FastGCN:Fast Learning with Graph Convolutional Networks via Importance Sampling[C]//International Conference on Learning Representations.2018.
[8]DEFFERRARD M,BRESSON X,VANDERGHEYNST P.Convolutional neural networks on graphs with fast localized spectral filtering[C]//Advances in Neural Information Processing Systems.2016:3844-3852.
[9]ZHANG Y,LIU Q,SONG L.Sentence-State LSTM for Text Representation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2018:317-327.
[10]WANG Y,HE Y M,CHEN H X,et al.RHS-CNN:A CNN Text Classification Model Based on Regularized Hierarchical Softmax[J].Journal of Chongqing University of Technology( Natural Science),2020,34( 5):187-195.
[11]BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.
[12]JOACHIMS T.Text categorization with support vector ma-chines:Learning with many relevant features[C]//European Conference on Machine Learning.Berlin:Springer,1998:137-142.
[13]WANG X,CHEN R,JIA Y,et al.Short text classification using wikipedia concept based document representation[C]//2013 International Conference on Information Technology and Applications.IEEE,2013:471-474.
[14]MCCALLUM A,NIGAM K.A comparison of event models for naive bayes text classification[C]//AAAI-98 Workshop on Learning for Text Categorization.1998:41-48.
[15]PHAN X H,NGUYEN L M,HORIGUCHI S.Learning to classify short and sparse text & web with hidden topics from large-scale data collections[C]//Proceedings of the 17th International Conference on World Wide Web.2008:91-100.
[16]ZHANG X,ZHAO J,LECUN Y.Character-level convolutional networks for text classification[C]//Advances in Neural Information Processing Systems.2015:649-657.
[17]WANG P,XU B,XU J,et al.Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification[J].Neurocomputing,2016,174:806-814.
[18]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019,33:7370-7377.
[19]HUANG L,MA D,LI S,et al.Text Level Graph NeuralNetwork for Text Classification[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Proces-sing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).2019:3435-3441.
[20]HU L M,YANG T C,SHI C,et al.Heterogeneous graph attention networks for semi-supervised short text classification[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).2019:4823-4832.
[21]CHE W X,LI Z H,LIU T.Ltp:A chinese language technology platform[C]//Coling 2010:Demonstrations.2010:13-16.
[22]YUAN L C.A statistical model of syntactic analysis based ondependence relations[J].Journal of Central South University:Natural Science Edition,2009(6):1630-1635.
[23]HAO M,XU B,LIANG J Y,et al.Chinese Short Text Classification with Mutual-Attention Convolutional Neural Networks[J].ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP),2020,19(5):1-13.
[24]WU X H,CHEN L,WEI T T,et al.Sentiment analysis of Chinese short text based on Self-Attention and Bi-LSTM[J].Journal of Chinese Information Processing,2019,33(6):100-107.
[25]DEFFERRARD M,BRESSON X,VANDERGHEYNST P.Convolutional neural networks on graphs with fast localized spectral filtering[J].Advances in Neural Information Processing Systems,2016,29:3844-3852.
[1] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[2] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[3] LI Xiao-wei, SHU Hui, GUANG Yan, ZHAI Yi, YANG Zi-ji. Survey of the Application of Natural Language Processing for Resume Analysis [J]. Computer Science, 2022, 49(6A): 66-73.
[4] SHAO Xin-xin. TI-FastText Automatic Goods Classification Algorithm [J]. Computer Science, 2022, 49(6A): 206-210.
[5] LI Zi-yi, ZHOU Xia-bing, WANG Zhong-qing, ZHANG Min. Stance Detection Based on User Connection [J]. Computer Science, 2022, 49(5): 221-226.
[6] GAO Yue, FU Xiang-ling, OUYANG Tian-xiong, CHEN Song-ling, YAN Chen-wei. EEG Emotion Recognition Based on Spatiotemporal Self-Adaptive Graph ConvolutionalNeural Network [J]. Computer Science, 2022, 49(4): 30-36.
[7] LIU Shuo, WANG Geng-run, PENG Jian-hua, LI Ke. Chinese Short Text Classification Algorithm Based on Hybrid Features of Characters and Words [J]. Computer Science, 2022, 49(4): 282-287.
[8] LI Hao, ZHANG Lan, YANG Bing, YANG Hai-xiao, KOU Yong-qi, WANG Fei, KANG Yan. Fine-grained Sentiment Classification of Chinese Microblogs Combining Dual Weight Mechanismand Graph Convolutional Neural Network [J]. Computer Science, 2022, 49(3): 246-254.
[9] MIAO Qi-guang, XIN Wen-tian, LIU Ru-yi, XIE Kun, WANG Quan, YANG Zong-kai. Graph Convolutional Skeleton-based Action Recognition Method for Intelligent Behavior Analysis [J]. Computer Science, 2022, 49(2): 156-161.
[10] CHEN Zhi-yi, SUI Jie. DeepFM and Convolutional Neural Networks Ensembles for Multimodal Rumor Detection [J]. Computer Science, 2022, 49(1): 101-107.
[11] WANG Li-mei, ZHU Xu-guang, WANG De-jia, ZHANG Yong, XING Chun-xiao. Study on Judicial Data Classification Method Based on Natural Language Processing Technologies [J]. Computer Science, 2021, 48(8): 80-85.
[12] LIANG Hao-hong, GU Tian-long, BIN Chen-zhong, CHANG Liang. Combining User-end and Item-end Knowledge Graph Learning for Personalized Recommendation [J]. Computer Science, 2021, 48(5): 109-116.
[13] WU Yu, LI Zhou-jun. Survey on Retrieval-based Chatbots [J]. Computer Science, 2021, 48(12): 278-285.
[14] KANG Yan, XIE Si-yu, WANG Fei, KOU Yong-qi, XU Yu-long, WU Zhi-wei, LI Hao. Traffic Prediction Model Based on Dual Path Information Spatial-Temporal Graph Convolutional Network [J]. Computer Science, 2021, 48(11A): 46-51.
[15] YE Song-tao, ZHOU Yang-zheng, FAN Hong-jie, CHEN Zheng-lei. Joint Learning of Causality and Spatio-Temporal Graph Convolutional Network for Skeleton- based Action Recognition [J]. Computer Science, 2021, 48(11A): 130-135.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!