Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (8): 205-216.doi: 10.11896/jsjkx.210800064

• Artificial Intelligence • Previous Articles     Next Articles

Review of Text Classification Methods Based on Graph Convolutional Network

TAN Ying-ying, WANG Jun-li, ZHANG Chao-bo   

  1. Key Laboratory of Embedded System and Service Computing(Tongji University),Ministry of Education,Shanghai 201804,China
  • Received:2021-08-09 Revised:2021-10-19 Published:2022-08-02
  • About author:TAN Ying-ying,born in 1998,postgra-duate.Her main research interests include natural language processing and deep learning.
    WANG Jun-li,born in 1978,Ph.D,associate researcher.Her main research interests include text data analysis,deep learning and artificial intelligence.
  • Supported by:
    National Key Research and Development Project of China(2017YFA0700602) and National Natural Science Foundation of China(61672381).

PDF (PC)

3760

Abstract: Text classification is a common task in natural language processing,in which there are a lot of research and progress based on machine learning and deep learning.However,these traditional methods can only process Euclidean spatial data,and cannot express the semantic information of the document effectively.To break the traditional learning mode,many recent studies start to use graphs to represent complicated relationships among entities in the document,and explore graph convolutional neural network for text representation.This paper reviews the text classification methods based on graph convolutional network.Firstly,the background and principle of graph convolutional network are summarized.Then,text classification methods based on graph convolutional network are described in detail according to different types of graph-based networks.Meanwhile,it analyzes the limi-tation of graph convolutional network in the depth of networks,and introduces the latest developments of deep networks in text classification.Finally,the classification performance of models involved in this paper is compared through some experiments,and the challenges and future research direction in this field are discussed.

Key words: Graph attention network, Graph convolutional network, Non-Euclidean space, Over-smoothing, Text classification

CLC Number: 

  • TP391.1
[1]WANG J,WANG Z,ZHANG D,et al.Combining knowledge with deep convolutional neural networks for short text classification[C]//Proceedings of the 26th International Joint Confe-rence on Artificial Intelligence.2017:2915-2921.
[2]HU X J,LIU L,QIU N J.A Novel Spam Categorization Algorithm Based on Active Learning Method and Negative Selection Algorithm[J].Dian Zi Xue Bao/Acta Electronica Sinica,2018,46(1):203-209.
[3]MAAS A,DALY R E,PHAM P T,et al.Learning word vectors for sentiment analysis[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies.2011:142-150.
[4]SU J S,ZHANG B F,XU X.Advances in Machine Learning Based Text Categorization[J].Ruan Jian Xue Bao/Journal of Software,2006,17(9):1848-1859.
[5]WU Y J,LI J,SONG C F,et al.High Utility Neural Networks for Text Classification[J].Dian Zi Xue Bao:Acta Electronica Sinica,2020,48(2):279-284.
[6]COVER T,HART P.Nearest neighbor pattern classification[J].IEEE Transactions on Information Theory,1967,13(1):21-27.
[7]JOACHIMS T.Text categorization with support vector ma-chines:Learning with many relevant features[C]//European Conference on Machine Learning.Berlin:Springer,1998:137-142.
[8]WANG S I,MANNING C D.Baselines and bigrams:Simple,good sentiment and topic classification[C]//Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics.2012:90-94.
[9]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A convolutional neural network for modelling sentences[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.2014:655-665.
[10]ZHU X,SOBHANI P,GUO H.Long short-term memory over recursive structures[C]//Proceedings of the 32ndInternational Conference on Machine Learning.2015:1604-1612.
[11]KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.2014:1746-1751.
[12]ZHANG X,ZHAO J,LECUN Y.Character-level convolutional networks for text classification[C]//Advances in Neural Information Processing Systems.2015:649-657.
[13]LIU P,QIU X,HUANG X.Recurrent Neural Network for Text Classification with Multi-Task Learning[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence.2016:2873-2879.
[14]YANG Z,YANG D,DYER C,et al.Hierarchical Attention Networks for Document Classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1480-1489.
[15]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[J].arXiv:1409.0473,2014.
[16]XU B B,CEN K T,HUANG J J,et al.A Survey on Graph Convolutional Neural Network[J].Ji Suan Ji Xue Bao/Chinese Journal of Computers,2020,43(5):755-780.
[17]WU Z,PAN S,CHEN F,et al.A Comprehensive Survey onGraph Neural Networks[J].IEEE Transactions on Neural Networks and Learning Systems,2021,32(1):4-24.
[18]ZHANG Z,CUI P,ZHU W.Deep Learning on Graphs:A Survey[J].IEEE Transactions on Knowledge and Data Enginee-ring,2022,34(1):249-270.
[19]ZHOU J,CUI G,HU S,et al.Graph neural networks:A review of methods and applications[J].AI Open,2020,1:57-81.
[20]MINAEE S,KALCHBRENNER N,CAMBRIA E,et al.DeepLearning Based Text Classification:A Comprehensive Review[J].arXiv:2004.03705v1,2020.
[21]LI Q,PENG H,LI J,et al.A Survey on Text Classification:From Shallow to Deep Learning[J].arXiv:2008.00364,2020.
[22]KIPF T N,WELLING M.A PyTorch implementation of GCN[EB/OL].https://github.com/tkipf/pygcn.
[23]HAMILTON W,YING Z,LESKOVEC J.A PyTorch imple-mentation of GraphSage[EB/OL].https://github.com/williamleif/graphsage-simple/.
[24]VELIKOVI P,CUCURULL G,CASANOVA A,et al.A Tensorflow implementation of GAT[EB/OL].https://github.com/PetarV-/GAT.
[25]CHEN J,MA T,XIAO C.A Tensorflow implementation ofFastGCN[EB/OL].https://github.com/matenure/FastGCN.
[26]WANG M,ZHENG D,YE Z,et al.A PyTorch implementation of JKNet [EB/OL].https://github.com/dmlc/dgl/tree/master/examples/pytorch/jknet.
[27]WU F,ZHANG T,SOUZA A,et al.A PyTorch implementation of SGC[EB/OL].https://github.com/Tiiiger/SGC.
[28]YAO L,MAO C,LUO Y.A Tensorflow implementation of Text-gcn[EB/OL].https://github.com/yao8839836/text_gcn.
[29]GAO H,CHEN Y,JI S.A Tensorflow implementation ofhConv-gPool-Net[EB/OL].https://github.com/HongyangGao/hConv-gPool-Net.
[30]KLICPERA J,BOJCHEVSKI A,GUNNEMANN S.Tensor-Flow and PyTorch implementations of PPNP and APPNP[EB/OL].https://github.com/klicperajo/ppnp.
[31]HUANG L,MA D,LI S,et al.A PyTorch implementation of Text-level GCN[EB/OL].https://github.com/mojave-pku/TextLevelGCN.
[32]HU L,YANG T,SHI C,et al.A PyTorch implementation ofHAGT[EB/OL].https://github.com/ytc272098215/HGAT.
[33]LIU X,YOU X,ZHANG X,et al.A Tensorflow implementation of TensorGCN[EB/OL].https://github.com/THUMLP/TensorGCN.
[34]RONG Y,HUANG W,XU T,et al.A PyTorch implementation of DropEdge[EB/OL].https://github.com/DropEdge/Drop-Edge.
[35]ZHANG J,MENG L.A PyTorch implementation of GResNet[EB/OL].https://github.com/jwzhanggy/GResNet.
[36]ZHAO L,AKOGLU L.A PyTorch implementation of PairNorm[EB/OL].https://github.com/LingxiaoShawn/PairNorm.
[37]CHEN G,TIAN Y,SONG Y.A PyTorch implementation ofD-GCN[EB/OL].https://github.com/cuhksz-nlp/DGSA.
[38]DING K,WANG J,LI J,et al.A PyTorch implementation of HyperGAT[EB/OL].https://github.com/kaize0409/HyperGAT_TextClassification.
[39]ZHANG Y,YU X,CUI Z,et al.A Tensorflow implementation of TextING[EB/OL].https://github.com/CRIPAC-DIG/Text-ING.
[40]YUAN L,WANG J,YU L,et al.A PyTorch implementation of G-ATT[EB/OL].https://github.com/YuanLi95/GATT-For-Aspect.
[41]FENG W,ZHANG J,DONG Y,et al.A PyTorch implementation of GRAND[EB/OL].https://github.com/THUDM/GRAND.
[42]ZHOU K,HUANG X,LI Y,et al.A PyTorch implementation of DGN[EB/OL].https://github.com/Kaixiong-Zhou/DGN.
[43]CHEN M,WEI Z,HUANG Z,et al.A PyTorch implementation of GCNII[EB/OL].https://github.com/chennnM/GCNII.
[44]BRUNA J,ZAREMBA W,SZLAM A,et al.Spectral networks and locally connected networks on graphs[J].arXiv:1312.6203,2013.
[45]CHUNG F R K.Spectral Graph Theory[EB/OL].http://www.ams.org/books/cbms/092/.
[46]SANDRYHAILA A,MOURA J M F.Discrete Signal Proces-sing on Graphs[J].IEEE Trans.on Signal Processing,2014,62(12):3042-3054.
[47]SHUMAN DI,NARANG S K,FROSSARD P,et al.The Emerging Field of Signal Processing on Graphs:Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains[J].IEEE Signal Processing Magazine,2013,30(3):83-98.
[48]DEFFERRARD M,BRESSON X,VANDERGHEYNST P.Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering[C]//Advances in Neural Information Processing Systems.2016:3844-3852.
[49]HAMMOND D K,VANDERGHEYNST P,GRIBONVAL R.Wavelets on graphs via spectral graph theory[J].Applied and Computational Harmonic Analysis,2011,30(2):129-150.
[50]KIPF T N,WELLING M.Semi-Supervised Classification withGraph Convolutional Networks[C]//Proceedings of the 5th International Conference on Learning Representations.2017:1-13.
[51]LI Y,TARLOW D,BROCKSCHMIDT M,et al.Gated graph sequence neural networks[J].arXiv:1511.05493,2016.
[52]YUN S,JEONG M,KIM R,et al.Graph Transformer Networks[C]//Advances in Neural Information Processing Systems.2019:11960-11970.
[53]YAO L,MAO C,LUO Y.Graph Convolutional Networks for Text Classification[C]//Proceedings of the 33rd AAAI Confe-rence on Artificial Intelligence.2019:7370-7377.
[54]CHURCH K W,HANKS P.Word association norms,mutual information,and lexicography[J].Computational linguistics,1990,16(1):22-29.
[55]ZAANEN M,KANTERS P.Automatic Mood Classificationusing TF*IDF based on Lyrics[C]//Proceedings of the 11th International Society for Music Information Retrieval Confe-rence.2010:75-80.
[56]LIU X,YOU X,ZHANG X,et al.Tensor graph convolutional networks for text classification[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence.2020:8409-8416.
[57]WU F,ZHANG T,SOUZA A,et al.Simplifying graph convolutional networks[C]//Proceedings of 36th International Confe-rence on Machine Learning.2019:6861-6871.
[58]HUANG L,MA D,LI S,et al.Text Level Graph Neural Network for Text Classification[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing.2019:3442-3448.
[59]GILMER J,SCHOENHOLZ S S,RILEY P F,et al.Neural message passing for quantum chemistry[C]//Proceedings of the 34th International Conference on Machine Learning.2017:1263-1272.
[60]GAO H,CHEN Y,JI S.Learning graph pooling and hybrid convolutional operations for text representations[C]//Proceedings of the World Wide Web Conference.2019:2743-2749.
[61]VELIKOVI P,CUCURULL G,CASANOVA A,et al.GraphAttention Networks[J].arXiv:1710.10903,2018.
[62]HU L,YANG T,SHI C,et al.Heterogeneous Graph AttentionNetworks for Semi-supervised Short Text Classification[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processingand the 9th International Joint Conference on Natural Language Processing.2019:4821-4830.
[63]YUAN L,WANG J,YU L,et al.Graph Attention Networkwith Memory Fusion for Aspect-level Sentiment Analysis[C]//Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing.2020:27-36.
[64]CHEN G,TIAN Y,SONG Y.Joint Aspect Extraction and Sentiment Analysis with Directional Graph Convolutional Networks[C]//Proceedings of the 28th International Conference on Computational Linguistics.2020:272-279.
[65]DING K,WANG J,LI J,et al.Be More with Less:Hypergraph Attention Networks for Inductive Text Classification[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.2020:4927-4936.
[66]BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.
[67]ZHANG Y,YU X,CUI Z,et al.Every Document Owns ItsStructure:Inductive Text Classification via Graph Neural Networks[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics.2020:334-339.
[68]YUE Z,QI L,SONG L.Sentence-State LSTM for Text Representation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.2018:317-327.
[69]LI W,LI S,MA S,et al.Recursive Graphical Neural Networks for Text Classification[J].arXiv:1909.08166,2020.
[70]HUANG B,CARLEY K M.Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing.2019:5469-5477.
[71]ZHANG H,ZHANG J.Text Graph Transformer for Document Classification[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.2020:8322-8327.
[72]LI Q,HAN Z,WU X.Deeper insights into graph convolutional networks for semi-supervised learning[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence.2018:3538-3545.
[73]KLICPERA J,BOJCHEVSKI A,GUNNEMANN S.PredictThen Propagate:Graph Neural Networks Meet Personalized Pagerank[J].arXiv:1810.05997,2019.
[74]PAGE L,BRIN S,MOTWANI R,et al.The pagerank citation ranking:Bringing order to the web[EB/OL].http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf.
[75]RONG Y,HUANG W,XU T,et al.DropEdge:Towards Deep Graph Convolutional Networks on Node Classification[C]//Proceedings of the International Conference on Learning Representations.2020:1-17.
[76]FENG W,ZHANG J,DONG Y,et al.Graph Random NeuralNetwork for Semi-Supervised Learning on Graphs[C]//Advances in Neural Information Processing Systems.2020:22092-22103.
[77]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Computer Socie-ty Conference on Computer Vision and Pattern Recognition.2016:770-778.
[78]XU K,LI C,TIAN Y,et al.Representation Learning on Graphs with Jumping Knowledge Networks[C]//Proceedings of the 35th International Conference on Machine Learning.2018:5449-5458.
[79]ZHANG J,MENG L.GResNet:GraphResidual Network for Reviving Deep GNNs from Suspended Animation[J].arXiv:1909.05729,2020.
[80]CHEN M,WEI Z,HUANG Z,et al.Simple and Deep Graph Convolutional Networks[C]//Proceedings of 37th International Conference on Machine Learning.2020:1725-1735.
[81]HEN D,LIN Y,LI W,et al.Measuring and relieving the over-smoothing problem for graph neural networks from the topolo-gical view[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence.2020:3438-3445.
[82]DWIVEDI V,JOSHI C,LAURENT T,et al.Benchmarkinggraph neural networks[J].arXiv:2003.00982,2020.
[83]ZHAO L,AKOGLU L.Pairnorm:Tackling oversmoothing inGNNS[J].arXiv:1909.12223v1,2020.
[84]ZHOU K,HUANG X,LI Y,et al.Towards Deeper Graph Neural Networks with Differentiable Group Normalization[C]//Advances in Neural Information Processing Systems Conference.2020:1-12.
[85]SEN P,NAMATA G,BILGIC M,et al.Collective classification in network data[J].AI magazine,2008,29(3):93-106.
[86]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]//Advances in Neural Information Processing Systems.2017:1024-1034.
[87]CHEN J,MA T,XIAO C.FastGCN:fast learning with graph convolutional networks via importance sampling[J].arXiv:1801.10247,2018.
[88]YANG Z,COHEN W,SALAKHUTDINOV R.Revisiting semi-supervised learning with graph embeddings[C]//Proceedings of the International Conference on Machine Learning.2016:40-48.
[1] WU Hong-xin, HAN Meng, CHEN Zhi-qiang, ZHANG Xi-long, LI Mu-hang. Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning [J]. Computer Science, 2022, 49(8): 12-25.
[2] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[3] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[4] SHI Dian-xi, ZHAO Chen-ran, ZHANG Yao-wen, YANG Shao-wu, ZHANG Yong-jun. Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning [J]. Computer Science, 2022, 49(8): 247-256.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] DENG Kai, YANG Pin, LI Yi-zhou, YANG Xing, ZENG Fan-rui, ZHANG Zhen-yu. Fast and Transmissible Domain Knowledge Graph Construction Method [J]. Computer Science, 2022, 49(6A): 100-108.
[7] KANG Yan, WU Zhi-wei, KOU Yong-qi, ZHANG Lan, XIE Si-yu, LI Hao. Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution [J]. Computer Science, 2022, 49(6A): 150-158.
[8] SHAO Xin-xin. TI-FastText Automatic Goods Classification Algorithm [J]. Computer Science, 2022, 49(6A): 206-210.
[9] ZHAO Xiao-hu, YE Sheng, LI Xiao. Multi-algorithm Fusion Behavior Classification Method for Body Bone Information Reconstruction [J]. Computer Science, 2022, 49(6): 269-275.
[10] DENG Zhao-yang, ZHONG Guo-qiang, WANG Dong. Text Classification Based on Attention Gated Graph Neural Network [J]. Computer Science, 2022, 49(6): 326-334.
[11] LIU Shuo, WANG Geng-run, PENG Jian-hua, LI Ke. Chinese Short Text Classification Algorithm Based on Hybrid Features of Characters and Words [J]. Computer Science, 2022, 49(4): 282-287.
[12] ZHONG Gui-feng, PANG Xiong-wen, SUI Dong. Text Classification Method Based on Word2Vec and AlexNet-2 with Improved AttentionMechanism [J]. Computer Science, 2022, 49(4): 288-293.
[13] ZHOU Hai-yu, ZHANG Dao-qiang. Multi-site Hyper-graph Convolutional Neural Networks and Application [J]. Computer Science, 2022, 49(3): 129-133.
[14] DENG Wei-bin, ZHU Kun, LI Yun-bo, HU Feng. FMNN:Text Classification Model Fused with Multiple Neural Networks [J]. Computer Science, 2022, 49(3): 281-287.
[15] PAN Zhi-hao, ZENG Bi, LIAO Wen-xiong, WEI Peng-fei, WEN Song. Interactive Attention Graph Convolutional Networks for Aspect-based Sentiment Classification [J]. Computer Science, 2022, 49(3): 294-300.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.