Computer Science ›› 2024, Vol. 51 ›› Issue (7): 303-309.doi: 10.11896/jsjkx.230400164

• Artificial Intelligence • Previous Articles     Next Articles

Text Classification Method Based on Multi Graph Convolution and Hierarchical Pooling

WEI Ziang, PENG Jian, HUANG Feihu, JU Shenggen   

  1. College of Computer Science,Sichuan University,Chengdu 610041,China
  • Received:2023-04-25 Revised:2023-09-15 Online:2024-07-15 Published:2024-07-10
  • About author:WEI Ziang,born in 1999,postgraduate,is a student member of CCF(No.J8855G).His main research interests include graph neural network and natural language processing.
    PENG Jian,born in 1970,Ph.D,professor,Ph.D supervisor,is an outstanding member of CCF(No.22761D).His main research interests include big data and wireless sensor network.
  • Supported by:
    Key R & D Program of Sichuan Province,China (2022YFG0034,2023YFG0115) and Cooperative Program of Sichuan University and Yibin(2020CDYB-30).

Abstract: Text classification,as a critical task in natural language processing,aims to assign labels to input documents.The Co-occurrence relationship between words offers key perspectives on text characteristics and vocabulary distribution,while word embeddings supply rich semantic information,influencing global vocabulary interaction and potential semantic relationships.Previous research has struggled to adequately incorporate both aspects or has disproportionately emphasized one over the other.To address this issue,a novel method is proposed in this paper that adaptively fuses these two types of information,aiming to strike a balance that can improve model performance while considering both structural relationships and embedded information.The method begins by constructing text data into text co-occurrence graphs and text embedding graphs,reflecting the context structure and semantic embedding information respectively.Graph convolution is then utilized to enhance node embeddings.In the graph pooling layer,node embeddings are fused and nodes of higher importance are identified by employing a hierarchical pooling model,learning document level representations layer by layer.Furthermore,we introduce a gated fusion module to adaptively fuse the embeddings of the two graphs.The proposed approach is validated with extensive experiments on five publicly available text classification datasets,and the experimental results show the superior performance of the HTGNN model in text classification tasks.

Key words: Text classification, Graph neural network, Graph representation learning, Graph classification, Attention mechanism

CLC Number: 

  • TP183
[1]KOLANU N,BROWN A S,BEECH A,et al.Natural languageprocessing of radiology reports for the identification of patients with fracture[J].Archives of Osteoporosis,2021,16:1-8.
[2]JACOVI A,SHALOM O S,GOLDBERG Y.Understandingconvolutional neural networks for text classification[J].arXiv:1809.08037,2018.
[3]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A convolutional neural network for modelling sentences[J].arXiv:1404.2188,2014.
[4]MIKOLOV T,KARAFIÁT M,BURGET L,et al.Recurrentneural network based language model [C]//Interspeech.2010:1045-1048.
[5]YANG Z,YANG D,DYER C,et al.Hierarchical attention networks for document classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1480-1489.
[6]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems,2017:6000-6010.
[7]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:7370-7377.
[8]HUANG L,MA D,LI S,et al.Text level graph neural network for text classification[J].arXiv:1910.02356,2019.
[9]WANG Y,WANG S,YAO Q,et al.Hierarchical heterogeneous graph representation learning for short text classification[J].arXiv:2111.00180,2021.
[10]CHEN Y Z,LIU X S,SUN L T,et al.Social Network Influence Prediction Algorithm Based on Graph Neural Network[J].Journal of Nanjing University(Natural Science),2022,58(3):386-397.
[11]MA Y Q,CAI M L,CHEN M,et al.Drug Interaction Prediction Method Based on Graph Neural Network[J].Computer Know-ledge and Technology,2022,18(18):61-63.
[12]WANG S,LI Y,ZHANG J,et al.Pm2.5-gnn:A domain know-ledge enhanced graph neural network for pm2.5 forecasting [C]//Proceedings of the 28th International Conference on Advances in Geographic Information Systems.2020:163-166.
[13]GILMER J,SCHOENHOLZ S S,RILEY P F,et al.Neural message passing for quantum chemistry[C]//International Confe-rence on Cachine Learning.PMLR,2017:1263-1272.
[14]ZHANG M,CUI Z,NEUMANN M,et al.An end-to-end deep learning architecture for graph classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[15]YING Z,YOU J,MORRIS C,et al.Hierarchical graph representation learning with differentiable pooling[C]//Advances in Neural Information Processing Systems,2018:4805-4815.
[16]MA Y,WANG S,AGGARWAL C C,et al.Graph convolutional networks with eigenpooling [C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining.2019:723-731.
[17]LEE J,LEE I,KANG J.Self-attention graph pooling [C]//International Conference on Machine Learning.PMLR,2019:3734-3743.
[18]ZHU Z L,RAO Y,WU Y,et al.Research Progress of Attention Mechanism in Deep Learning[J].Journal of Chinese Information Processing,2019,33(6):1-11.
[19]LIU G,GUO J.Bidirectional LSTM with attention mechanism and convolutional layer for text classification[J].Neurocompu-ting,2019,337:325338.
[20]SHEN T,ZHOU T,LONG G,et al.Disan:Directional self-attention network for rnn/cnn-free language understanding [C]//Proceedings of the AAAI conference on Artificial Intelligence.2018.
[21]VELIKOVI P,CUCURULL G,CASANOVA A,et al.Graph attention networks[J].arXiv:1710.10903,2017.
[22]ZHANG J,SHI X,XIE J,et al.Gaan:Gated attention networks for learning on large and spatiotemporal graphs[J].arXiv:1803.07294,2018.
[23]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[24]GLOROT X,BENGIO Y.Understanding the difficulty of trai-ning deep feedforward neural networks[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics.JMLR Workshop and Conference Proceedings,2010:249-256.
[25]HINTON G E,SRIVASTAVA N,KRIZHEVSKY A,et al.Improving neural networks by preventing co-adaptation of feature detectors[J].arXiv:1207.0580,2012.
[26]JOULIN A,GRAVE E,BOJANOWSKI P,et al.Bag of tricks for efficient text classification[J].arXiv:1607.01759,2016.
[27]SHEN D,WANG G,WANG W,et al.Baseline needs more love:On simple word-embedding-based models and associated pooling mechanisms[J].arXiv:1805.09843,2018.
[28]LIU X,YOU X,ZHANG X,et al.Tensor graph convolutional networks for text classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:8409-8416.
[29]RANJAN E,SANYAL S,TALUKDAR P.Asap:Adaptivestructure aware pooling for learning hierarchical graph representations [C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:5470-5477.
[30]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[31]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]//Advances in Neural Information Processing Systems.2017:1025-1035.
[32]QIAO L,ZHANG L,CHEN S,et al.Data-driven graph construction and graph learning:A review[J].Neurocomputing,2018,312:336-351.
[1] BAI Wenchao, BAI Shuwen, HAN Xixian, ZHAO Yubo. Efficient Query Workload Prediction Algorithm Based on TCN-A [J]. Computer Science, 2024, 51(7): 71-79.
[2] ZENG Zihui, LI Chaoyang, LIAO Qing. Multivariate Time Series Anomaly Detection Algorithm in Missing Value Scenario [J]. Computer Science, 2024, 51(7): 108-115.
[3] YANG Zhenzhen, WANG Dongtao, YANG Yongpeng, HUA Renyu. Multi-embedding Fusion Based on top-N Recommendation [J]. Computer Science, 2024, 51(7): 140-145.
[4] HU Haibo, YANG Dan, NIE Tiezheng, KOU Yue. Graph Contrastive Learning Incorporating Multi-influence and Preference for Social Recommendation [J]. Computer Science, 2024, 51(7): 146-155.
[5] LI Jiaying, LIANG Yudong, LI Shaoji, ZHANG Kunpeng, ZHANG Chao. Study on Algorithm of Depth Image Super-resolution Guided by High-frequency Information ofColor Images [J]. Computer Science, 2024, 51(7): 197-205.
[6] LOU Zhengzheng, ZHANG Xin, HU Shizhe, WU Yunpeng. Foggy Weather Object Detection Method Based on YOLOX_s [J]. Computer Science, 2024, 51(7): 206-213.
[7] YAN Jingtao, LI Yang, WANG Suge, PAN Bangze. Overlap Event Extraction Method with Language Granularity Fusion Based on Joint Learning [J]. Computer Science, 2024, 51(7): 287-295.
[8] WANG Xianwei, FENG Xiang, YU Huiqun. Multi-agent Cooperative Algorithm for Obstacle Clearance Based on Deep Deterministic PolicyGradient and Attention Critic [J]. Computer Science, 2024, 51(7): 319-326.
[9] FAN Yi, HU Tao, YI Peng. Host Anomaly Detection Framework Based on Multifaceted Information Fusion of SemanticFeatures for System Calls [J]. Computer Science, 2024, 51(7): 380-388.
[10] ZHANG Le, YU Ying, GE Hao. Mural Inpainting Based on Fast Fourier Convolution and Feature Pruning Coordinate Attention [J]. Computer Science, 2024, 51(6A): 230400083-9.
[11] SUN Yang, DING Jianwei, ZHANG Qi, WEI Huiwen, TIAN Bowen. Study on Super-resolution Image Reconstruction Using Residual Feature Aggregation NetworkBased on Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230600039-6.
[12] QUE Yue, GAN Menghan, LIU Zhiwei. Object Detection with Receptive Field Expansion and Multi-branch Aggregation [J]. Computer Science, 2024, 51(6A): 230600151-6.
[13] HE Xinyu, LU Chenxin, FENG Shuyi, OUYANG Shangrong, MU Wentao. Ship Detection and Recognition of Optical Remote Sensing Images for Embedded Platform [J]. Computer Science, 2024, 51(6A): 230700117-7.
[14] LI Guo, CHEN Chen, YANG Jing, QUN Nuo. Study on Tibetan Short Text Classification Based on DAN and FastText [J]. Computer Science, 2024, 51(6A): 230700064-5.
[15] HUANG Rui, XU Ji. Text Classification Based on Invariant Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230900018-5.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!