计算机科学 ›› 2022, Vol. 49 ›› Issue (8): 205-216.doi: 10.11896/jsjkx.210800064

• 人工智能 • 上一篇    下一篇

基于图卷积神经网络的文本分类方法研究综述

檀莹莹, 王俊丽, 张超波   

  1. 嵌入式系统与服务计算教育部重点实验室(同济大学) 上海 201804
  • 收稿日期:2021-08-09 修回日期:2021-10-19 发布日期:2022-08-02
  • 通讯作者: 王俊丽(junliwang@tongji.edu.cn)
  • 作者简介:(tansyka@tongji.edu.cn)
  • 基金资助:
    国家重点研发计划(2017YFA0700602);国家自然科学基金(61672381)

Review of Text Classification Methods Based on Graph Convolutional Network

TAN Ying-ying, WANG Jun-li, ZHANG Chao-bo   

  1. Key Laboratory of Embedded System and Service Computing(Tongji University),Ministry of Education,Shanghai 201804,China
  • Received:2021-08-09 Revised:2021-10-19 Published:2022-08-02
  • About author:TAN Ying-ying,born in 1998,postgra-duate.Her main research interests include natural language processing and deep learning.
    WANG Jun-li,born in 1978,Ph.D,associate researcher.Her main research interests include text data analysis,deep learning and artificial intelligence.
  • Supported by:
    National Key Research and Development Project of China(2017YFA0700602) and National Natural Science Foundation of China(61672381).

摘要: 文本分类是自然语言处理领域中常见的任务,机器学习和深度学习在该任务中已有较多研究并取得了很大进展,然而,这些传统方法只能处理欧氏空间的数据,不能完全有效地表达出文本的语义信息。为了打破传统的学习模式,诸多研究开始尝试用图表示文本中各实体间的丰富关系,并利用图卷积神经网络学习文本表示。文中对基于图卷积神经网络的文本分类方法进行了综述,首先概述了图卷积神经网络的背景与原理;其次,利用不同类型的图网络详细阐述了基于图卷积神经网络的文本分类方法,同时分析了图卷积神经网络在网络深度上的局限性,并介绍了深层网络在文本分类任务上的最新进展;最后,通过实验比较了各模型的分类性能,并探讨了该领域的难点与未来的发展方向。

关键词: 非欧空间, 过平滑, 图卷积神经网络, 图注意力网络, 文本分类

Abstract: Text classification is a common task in natural language processing,in which there are a lot of research and progress based on machine learning and deep learning.However,these traditional methods can only process Euclidean spatial data,and cannot express the semantic information of the document effectively.To break the traditional learning mode,many recent studies start to use graphs to represent complicated relationships among entities in the document,and explore graph convolutional neural network for text representation.This paper reviews the text classification methods based on graph convolutional network.Firstly,the background and principle of graph convolutional network are summarized.Then,text classification methods based on graph convolutional network are described in detail according to different types of graph-based networks.Meanwhile,it analyzes the limi-tation of graph convolutional network in the depth of networks,and introduces the latest developments of deep networks in text classification.Finally,the classification performance of models involved in this paper is compared through some experiments,and the challenges and future research direction in this field are discussed.

Key words: Graph attention network, Graph convolutional network, Non-Euclidean space, Over-smoothing, Text classification

中图分类号: 

  • TP391.1
[1]WANG J,WANG Z,ZHANG D,et al.Combining knowledge with deep convolutional neural networks for short text classification[C]//Proceedings of the 26th International Joint Confe-rence on Artificial Intelligence.2017:2915-2921.
[2]HU X J,LIU L,QIU N J.A Novel Spam Categorization Algorithm Based on Active Learning Method and Negative Selection Algorithm[J].Dian Zi Xue Bao/Acta Electronica Sinica,2018,46(1):203-209.
[3]MAAS A,DALY R E,PHAM P T,et al.Learning word vectors for sentiment analysis[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies.2011:142-150.
[4]SU J S,ZHANG B F,XU X.Advances in Machine Learning Based Text Categorization[J].Ruan Jian Xue Bao/Journal of Software,2006,17(9):1848-1859.
[5]WU Y J,LI J,SONG C F,et al.High Utility Neural Networks for Text Classification[J].Dian Zi Xue Bao:Acta Electronica Sinica,2020,48(2):279-284.
[6]COVER T,HART P.Nearest neighbor pattern classification[J].IEEE Transactions on Information Theory,1967,13(1):21-27.
[7]JOACHIMS T.Text categorization with support vector ma-chines:Learning with many relevant features[C]//European Conference on Machine Learning.Berlin:Springer,1998:137-142.
[8]WANG S I,MANNING C D.Baselines and bigrams:Simple,good sentiment and topic classification[C]//Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics.2012:90-94.
[9]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A convolutional neural network for modelling sentences[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.2014:655-665.
[10]ZHU X,SOBHANI P,GUO H.Long short-term memory over recursive structures[C]//Proceedings of the 32ndInternational Conference on Machine Learning.2015:1604-1612.
[11]KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.2014:1746-1751.
[12]ZHANG X,ZHAO J,LECUN Y.Character-level convolutional networks for text classification[C]//Advances in Neural Information Processing Systems.2015:649-657.
[13]LIU P,QIU X,HUANG X.Recurrent Neural Network for Text Classification with Multi-Task Learning[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence.2016:2873-2879.
[14]YANG Z,YANG D,DYER C,et al.Hierarchical Attention Networks for Document Classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1480-1489.
[15]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[J].arXiv:1409.0473,2014.
[16]XU B B,CEN K T,HUANG J J,et al.A Survey on Graph Convolutional Neural Network[J].Ji Suan Ji Xue Bao/Chinese Journal of Computers,2020,43(5):755-780.
[17]WU Z,PAN S,CHEN F,et al.A Comprehensive Survey onGraph Neural Networks[J].IEEE Transactions on Neural Networks and Learning Systems,2021,32(1):4-24.
[18]ZHANG Z,CUI P,ZHU W.Deep Learning on Graphs:A Survey[J].IEEE Transactions on Knowledge and Data Enginee-ring,2022,34(1):249-270.
[19]ZHOU J,CUI G,HU S,et al.Graph neural networks:A review of methods and applications[J].AI Open,2020,1:57-81.
[20]MINAEE S,KALCHBRENNER N,CAMBRIA E,et al.DeepLearning Based Text Classification:A Comprehensive Review[J].arXiv:2004.03705v1,2020.
[21]LI Q,PENG H,LI J,et al.A Survey on Text Classification:From Shallow to Deep Learning[J].arXiv:2008.00364,2020.
[22]KIPF T N,WELLING M.A PyTorch implementation of GCN[EB/OL].https://github.com/tkipf/pygcn.
[23]HAMILTON W,YING Z,LESKOVEC J.A PyTorch imple-mentation of GraphSage[EB/OL].https://github.com/williamleif/graphsage-simple/.
[24]VELIKOVI P,CUCURULL G,CASANOVA A,et al.A Tensorflow implementation of GAT[EB/OL].https://github.com/PetarV-/GAT.
[25]CHEN J,MA T,XIAO C.A Tensorflow implementation ofFastGCN[EB/OL].https://github.com/matenure/FastGCN.
[26]WANG M,ZHENG D,YE Z,et al.A PyTorch implementation of JKNet [EB/OL].https://github.com/dmlc/dgl/tree/master/examples/pytorch/jknet.
[27]WU F,ZHANG T,SOUZA A,et al.A PyTorch implementation of SGC[EB/OL].https://github.com/Tiiiger/SGC.
[28]YAO L,MAO C,LUO Y.A Tensorflow implementation of Text-gcn[EB/OL].https://github.com/yao8839836/text_gcn.
[29]GAO H,CHEN Y,JI S.A Tensorflow implementation ofhConv-gPool-Net[EB/OL].https://github.com/HongyangGao/hConv-gPool-Net.
[30]KLICPERA J,BOJCHEVSKI A,GUNNEMANN S.Tensor-Flow and PyTorch implementations of PPNP and APPNP[EB/OL].https://github.com/klicperajo/ppnp.
[31]HUANG L,MA D,LI S,et al.A PyTorch implementation of Text-level GCN[EB/OL].https://github.com/mojave-pku/TextLevelGCN.
[32]HU L,YANG T,SHI C,et al.A PyTorch implementation ofHAGT[EB/OL].https://github.com/ytc272098215/HGAT.
[33]LIU X,YOU X,ZHANG X,et al.A Tensorflow implementation of TensorGCN[EB/OL].https://github.com/THUMLP/TensorGCN.
[34]RONG Y,HUANG W,XU T,et al.A PyTorch implementation of DropEdge[EB/OL].https://github.com/DropEdge/Drop-Edge.
[35]ZHANG J,MENG L.A PyTorch implementation of GResNet[EB/OL].https://github.com/jwzhanggy/GResNet.
[36]ZHAO L,AKOGLU L.A PyTorch implementation of PairNorm[EB/OL].https://github.com/LingxiaoShawn/PairNorm.
[37]CHEN G,TIAN Y,SONG Y.A PyTorch implementation ofD-GCN[EB/OL].https://github.com/cuhksz-nlp/DGSA.
[38]DING K,WANG J,LI J,et al.A PyTorch implementation of HyperGAT[EB/OL].https://github.com/kaize0409/HyperGAT_TextClassification.
[39]ZHANG Y,YU X,CUI Z,et al.A Tensorflow implementation of TextING[EB/OL].https://github.com/CRIPAC-DIG/Text-ING.
[40]YUAN L,WANG J,YU L,et al.A PyTorch implementation of G-ATT[EB/OL].https://github.com/YuanLi95/GATT-For-Aspect.
[41]FENG W,ZHANG J,DONG Y,et al.A PyTorch implementation of GRAND[EB/OL].https://github.com/THUDM/GRAND.
[42]ZHOU K,HUANG X,LI Y,et al.A PyTorch implementation of DGN[EB/OL].https://github.com/Kaixiong-Zhou/DGN.
[43]CHEN M,WEI Z,HUANG Z,et al.A PyTorch implementation of GCNII[EB/OL].https://github.com/chennnM/GCNII.
[44]BRUNA J,ZAREMBA W,SZLAM A,et al.Spectral networks and locally connected networks on graphs[J].arXiv:1312.6203,2013.
[45]CHUNG F R K.Spectral Graph Theory[EB/OL].http://www.ams.org/books/cbms/092/.
[46]SANDRYHAILA A,MOURA J M F.Discrete Signal Proces-sing on Graphs[J].IEEE Trans.on Signal Processing,2014,62(12):3042-3054.
[47]SHUMAN DI,NARANG S K,FROSSARD P,et al.The Emerging Field of Signal Processing on Graphs:Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains[J].IEEE Signal Processing Magazine,2013,30(3):83-98.
[48]DEFFERRARD M,BRESSON X,VANDERGHEYNST P.Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering[C]//Advances in Neural Information Processing Systems.2016:3844-3852.
[49]HAMMOND D K,VANDERGHEYNST P,GRIBONVAL R.Wavelets on graphs via spectral graph theory[J].Applied and Computational Harmonic Analysis,2011,30(2):129-150.
[50]KIPF T N,WELLING M.Semi-Supervised Classification withGraph Convolutional Networks[C]//Proceedings of the 5th International Conference on Learning Representations.2017:1-13.
[51]LI Y,TARLOW D,BROCKSCHMIDT M,et al.Gated graph sequence neural networks[J].arXiv:1511.05493,2016.
[52]YUN S,JEONG M,KIM R,et al.Graph Transformer Networks[C]//Advances in Neural Information Processing Systems.2019:11960-11970.
[53]YAO L,MAO C,LUO Y.Graph Convolutional Networks for Text Classification[C]//Proceedings of the 33rd AAAI Confe-rence on Artificial Intelligence.2019:7370-7377.
[54]CHURCH K W,HANKS P.Word association norms,mutual information,and lexicography[J].Computational linguistics,1990,16(1):22-29.
[55]ZAANEN M,KANTERS P.Automatic Mood Classificationusing TF*IDF based on Lyrics[C]//Proceedings of the 11th International Society for Music Information Retrieval Confe-rence.2010:75-80.
[56]LIU X,YOU X,ZHANG X,et al.Tensor graph convolutional networks for text classification[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence.2020:8409-8416.
[57]WU F,ZHANG T,SOUZA A,et al.Simplifying graph convolutional networks[C]//Proceedings of 36th International Confe-rence on Machine Learning.2019:6861-6871.
[58]HUANG L,MA D,LI S,et al.Text Level Graph Neural Network for Text Classification[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing.2019:3442-3448.
[59]GILMER J,SCHOENHOLZ S S,RILEY P F,et al.Neural message passing for quantum chemistry[C]//Proceedings of the 34th International Conference on Machine Learning.2017:1263-1272.
[60]GAO H,CHEN Y,JI S.Learning graph pooling and hybrid convolutional operations for text representations[C]//Proceedings of the World Wide Web Conference.2019:2743-2749.
[61]VELIKOVI P,CUCURULL G,CASANOVA A,et al.GraphAttention Networks[J].arXiv:1710.10903,2018.
[62]HU L,YANG T,SHI C,et al.Heterogeneous Graph AttentionNetworks for Semi-supervised Short Text Classification[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processingand the 9th International Joint Conference on Natural Language Processing.2019:4821-4830.
[63]YUAN L,WANG J,YU L,et al.Graph Attention Networkwith Memory Fusion for Aspect-level Sentiment Analysis[C]//Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing.2020:27-36.
[64]CHEN G,TIAN Y,SONG Y.Joint Aspect Extraction and Sentiment Analysis with Directional Graph Convolutional Networks[C]//Proceedings of the 28th International Conference on Computational Linguistics.2020:272-279.
[65]DING K,WANG J,LI J,et al.Be More with Less:Hypergraph Attention Networks for Inductive Text Classification[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.2020:4927-4936.
[66]BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.
[67]ZHANG Y,YU X,CUI Z,et al.Every Document Owns ItsStructure:Inductive Text Classification via Graph Neural Networks[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics.2020:334-339.
[68]YUE Z,QI L,SONG L.Sentence-State LSTM for Text Representation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.2018:317-327.
[69]LI W,LI S,MA S,et al.Recursive Graphical Neural Networks for Text Classification[J].arXiv:1909.08166,2020.
[70]HUANG B,CARLEY K M.Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing.2019:5469-5477.
[71]ZHANG H,ZHANG J.Text Graph Transformer for Document Classification[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.2020:8322-8327.
[72]LI Q,HAN Z,WU X.Deeper insights into graph convolutional networks for semi-supervised learning[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence.2018:3538-3545.
[73]KLICPERA J,BOJCHEVSKI A,GUNNEMANN S.PredictThen Propagate:Graph Neural Networks Meet Personalized Pagerank[J].arXiv:1810.05997,2019.
[74]PAGE L,BRIN S,MOTWANI R,et al.The pagerank citation ranking:Bringing order to the web[EB/OL].http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf.
[75]RONG Y,HUANG W,XU T,et al.DropEdge:Towards Deep Graph Convolutional Networks on Node Classification[C]//Proceedings of the International Conference on Learning Representations.2020:1-17.
[76]FENG W,ZHANG J,DONG Y,et al.Graph Random NeuralNetwork for Semi-Supervised Learning on Graphs[C]//Advances in Neural Information Processing Systems.2020:22092-22103.
[77]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Computer Socie-ty Conference on Computer Vision and Pattern Recognition.2016:770-778.
[78]XU K,LI C,TIAN Y,et al.Representation Learning on Graphs with Jumping Knowledge Networks[C]//Proceedings of the 35th International Conference on Machine Learning.2018:5449-5458.
[79]ZHANG J,MENG L.GResNet:GraphResidual Network for Reviving Deep GNNs from Suspended Animation[J].arXiv:1909.05729,2020.
[80]CHEN M,WEI Z,HUANG Z,et al.Simple and Deep Graph Convolutional Networks[C]//Proceedings of 37th International Conference on Machine Learning.2020:1725-1735.
[81]HEN D,LIN Y,LI W,et al.Measuring and relieving the over-smoothing problem for graph neural networks from the topolo-gical view[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence.2020:3438-3445.
[82]DWIVEDI V,JOSHI C,LAURENT T,et al.Benchmarkinggraph neural networks[J].arXiv:2003.00982,2020.
[83]ZHAO L,AKOGLU L.Pairnorm:Tackling oversmoothing inGNNS[J].arXiv:1909.12223v1,2020.
[84]ZHOU K,HUANG X,LI Y,et al.Towards Deeper Graph Neural Networks with Differentiable Group Normalization[C]//Advances in Neural Information Processing Systems Conference.2020:1-12.
[85]SEN P,NAMATA G,BILGIC M,et al.Collective classification in network data[J].AI magazine,2008,29(3):93-106.
[86]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]//Advances in Neural Information Processing Systems.2017:1024-1034.
[87]CHEN J,MA T,XIAO C.FastGCN:fast learning with graph convolutional networks via importance sampling[J].arXiv:1801.10247,2018.
[88]YANG Z,COHEN W,SALAKHUTDINOV R.Revisiting semi-supervised learning with graph embeddings[C]//Proceedings of the International Conference on Machine Learning.2016:40-48.
[1] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[2] 武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航.
监督和半监督学习下的多标签分类综述
Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning
计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111
[3] 史殿习, 赵琛然, 张耀文, 杨绍武, 张拥军.
基于多智能体强化学习的端到端合作的自适应奖励方法
Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning
计算机科学, 2022, 49(8): 247-256. https://doi.org/10.11896/jsjkx.210700100
[4] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[5] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6] 邵欣欣.
TI-FastText自动商品分类算法
TI-FastText Automatic Goods Classification Algorithm
计算机科学, 2022, 49(6A): 206-210. https://doi.org/10.11896/jsjkx.210500089
[7] 邓凯, 杨频, 李益洲, 杨星, 曾凡瑞, 张振毓.
一种可快速迁移的领域知识图谱构建方法
Fast and Transmissible Domain Knowledge Graph Construction Method
计算机科学, 2022, 49(6A): 100-108. https://doi.org/10.11896/jsjkx.210900018
[8] 康雁, 吴志伟, 寇勇奇, 张兰, 谢思宇, 李浩.
融合Bert和图卷积的深度集成学习软件需求分类
Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution
计算机科学, 2022, 49(6A): 150-158. https://doi.org/10.11896/jsjkx.210500065
[9] 邓朝阳, 仲国强, 王栋.
基于注意力门控图神经网络的文本分类
Text Classification Based on Attention Gated Graph Neural Network
计算机科学, 2022, 49(6): 326-334. https://doi.org/10.11896/jsjkx.210400218
[10] 李子仪, 周夏冰, 王中卿, 张民.
基于用户关联的立场检测
Stance Detection Based on User Connection
计算机科学, 2022, 49(5): 221-226. https://doi.org/10.11896/jsjkx.210400135
[11] 高越, 傅湘玲, 欧阳天雄, 陈松龄, 闫晨巍.
基于时空自适应图卷积神经网络的脑电信号情绪识别
EEG Emotion Recognition Based on Spatiotemporal Self-Adaptive Graph ConvolutionalNeural Network
计算机科学, 2022, 49(4): 30-36. https://doi.org/10.11896/jsjkx.210900200
[12] 刘硕, 王庚润, 彭建华, 李柯.
基于混合字词特征的中文短文本分类算法
Chinese Short Text Classification Algorithm Based on Hybrid Features of Characters and Words
计算机科学, 2022, 49(4): 282-287. https://doi.org/10.11896/jsjkx.210200027
[13] 钟桂凤, 庞雄文, 隋栋.
基于Word2Vec和改进注意力机制AlexNet-2的文本分类方法
Text Classification Method Based on Word2Vec and AlexNet-2 with Improved AttentionMechanism
计算机科学, 2022, 49(4): 288-293. https://doi.org/10.11896/jsjkx.211100016
[14] 李浩, 张兰, 杨兵, 杨海潇, 寇勇奇, 王飞, 康雁.
融合双重权重机制和图卷积神经网络的微博细粒度情感分类
Fine-grained Sentiment Classification of Chinese Microblogs Combining Dual Weight Mechanismand Graph Convolutional Neural Network
计算机科学, 2022, 49(3): 246-254. https://doi.org/10.11896/jsjkx.201200073
[15] 邓维斌, 朱坤, 李云波, 胡峰.
FMNN:融合多神经网络的文本分类模型
FMNN:Text Classification Model Fused with Multiple Neural Networks
计算机科学, 2022, 49(3): 281-287. https://doi.org/10.11896/jsjkx.210200090
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!