Computer Science ›› 2024, Vol. 51 ›› Issue (3): 48-55.doi: 10.11896/jsjkx.221200158

• Database & Big Data & Data Science • Previous Articles     Next Articles

Framework and Algorithms for Accelerating Training of Semi-supervised Graph Neural Network Based on Heuristic Coarsening Algorithms

CHEN Yufeng , HUANG Zengfeng   

  1. School of Data Science,Fudan University,Shanghai 200433,China
  • Received:2022-12-27 Revised:2023-05-25 Online:2024-03-15 Published:2024-03-13
  • About author:CHEN Yufeng,born in 1999,postgra-duate.His main research interest is graph neural networks.HUANG Zengfeng,born in 1986,Ph.D,researcher,doctoral advisor.His main research interests include machine learning algorithms and theory,big data and theoretical computation computer science.

Abstract: Graph neural network is the mainstream tool of graph machine learning at the current stage,and it has broad development prospects.By constructing an abstract graph structure,the graph neural network model can be used to efficiently deal with problems in various application scenarios,including node prediction,link prediction,and graph classification.But the application on large-scale graphs has always been the key point and difficulty in graph neural network training.And how to effectively and quickly train and deploy graph neural networks on large-scale graph data is a major problem hindering the further industrial application of graph neural networks.Graph neural network can use the topological information of the network structure of the graph,so as to achieve better results than other general neural networks such as multi-layer perceptron on the node prediction problem.But the rapid growth of the number of nodes and edges of the graph's network structure restricts the training of the graph neural network,and the number of nodes in the real dataset is tens of millions or even billions,or the number of edges in some dense network structures has reached tens of millions.This makes it difficult for traditional graph neural network training methods to achieve direct results.This paper improves and proposes a new framework for graph neural network training based on heuristic graph coarsening algorithms,and proposes two specific training algorithms on this basis.Then this paper proposes two simple heuristic graph coarsening algorithms.Under the guarantee that the loss of accuracy is acceptable and the memory space consumption is greatly reduced,the proposed algorithm can further significantly reduce both calculation time and training time of graph neural networks.Experiment shows that satisfactory results can be achieved on common datasets.

Key words: Graph neural network, Graph coarsening, Training acceleration, Heuristic, Random walk, Unbiased

CLC Number: 

  • TP391
[1]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[2]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[J].Advances in Neural Information Processing Systems,2017:30,1024-1034.
[3]CHEN J,MA T,XIAO C.Fastgcn:fast learning with graph con-volutional networks via importance sampling[J].arXiv:1801.10247,2018.
[4]CHEN J,ZHU J,SONG L.Stochastic training of graph convolutional networks with variance reduction[J].arXiv:1710.10568,2017.
[5]CHANG W L,LIU X,SI S,et al.Cluster-gcn:An efficient algorithm for training deep and large graph convolutional networks[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2019:257-266.
[6]ZENG H,ZHOU H,SRIVASTAVA A,et al.Graphsaint:Graph sampling based inductive learning method[J].arXiv:1907.04931,2019.
[7]HUANG Z,ZHANG S,XI C,et al.Scaling up graph neural networks via graph coarsening[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining.2021:675-684.
[8]ANDERSEN R,CHUNG F,LANG K.Local graph partitioning using pagerank vectors[C]//2006 47th Annual IEEE Sympo-sium on Foundations of Computer Science(FOCS'06).IEEE,2006:475-486.
[9]LOUKAS A.Graph Reduction with Spectral and Cut Guarantees[J].Journal of Machine Learning Research,2019,20(116):1-42.
[10]LOUKAS A,VANDERGHEYNST P.Spectrally approximating large graphs with smaller graphs[C]//International Conference on Machine Learning.PMLR,2018:3237-3246.
[11]WU F,SOUZA A,ZHANG T,et al.Simplifying graph convolutional networks[C]//International Conference on Machine Learning.PMLR,2019:6861-6871.
[12]YANG Z,COHEN W,SALAKHUDINOV R.Revisiting semi-supervised learning with graph embeddings[C]//International Conference on Machine Learning.PMLR,2016:40-48.
[13]PEROZZI B,AL-RFOU R,SKIENA S.Deepwalk:Online lear-ning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2014:701-710.
[14]WANG M,ZHENG D,YE Z,et al.Deep graph library:A graph-centric,highly-performant package for graph neural networks[J].arXiv:1909.01315,2019.
[15]GOODFELLOW I,BENGIO Y,COURVILLE A.Deep learning[M].The MIT press,2016.
[16]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[17]BOJCHEVSKI A,KLICPERA J,PEROZZI B,et al.Scalinggraph neural networks with approximate pagerank[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2020:2464-2473.
[1] LUO Zeyang, TIAN Hua, DOU Yingtong, LI Manwen, ZHANG Zehua. Fake Review Detection Based on Residual Networks Fusion of Multi-relationship Review Features [J]. Computer Science, 2024, 51(4): 314-323.
[2] ZHANG Liying, SUN Haihang, SUN Yufa , SHI Bingbo. Review of Node Classification Methods Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(4): 95-105.
[3] ZHANG Tao, LIAO Bin, YU Jiong, LI Ming, SUN Ruina. Benchmarking and Analysis for Graph Neural Network Node Classification Task [J]. Computer Science, 2024, 51(4): 132-150.
[4] HUANG Kun, SUN Weiwei. Traffic Speed Forecasting Algorithm Based on Missing Data [J]. Computer Science, 2024, 51(3): 72-80.
[5] CHEN Wei, ZHOU Lihua, WANG Yafeng, WANG Lizhen, CHEN Hongmei. Community Search Based on Disentangled Graph Neural Network in Heterogeneous Information Networks [J]. Computer Science, 2024, 51(3): 90-101.
[6] ZHANG Guohao, WANG Yi, ZHOU Xi, WANG Baoquan. Deep Collaborative Truth Discovery Based on Variational Multi-hop Graph Attention Encoder [J]. Computer Science, 2024, 51(3): 109-117.
[7] ZHENG Cheng, SHI Jingwei, WEI Suhua, CHENG Jiaming. Dual Feature Adaptive Fusion Network Based on Dependency Type Pruning for Aspect-basedSentiment Analysis [J]. Computer Science, 2024, 51(3): 205-213.
[8] XU Tianyue, LIU Xianhui, ZHAO Weidong. Knowledge Graph and User Interest Based Recommendation Algorithm [J]. Computer Science, 2024, 51(2): 55-62.
[9] GUO Yuxing, YAO Kaixuan, WANG Zhiqiang, WEN Liangliang, LIANG Jiye. Black-box Graph Adversarial Attacks Based on Topology and Feature Fusion [J]. Computer Science, 2024, 51(1): 355-362.
[10] JIN Yu, CHEN Hongmei, LUO Chuan. Interest Capturing Recommendation Based on Knowledge Graph [J]. Computer Science, 2024, 51(1): 133-142.
[11] WU Jiawei, FANG Quan, HU Jun, QIAN Shengsheng. Pre-training of Heterogeneous Graph Neural Networks for Multi-label Document Classification [J]. Computer Science, 2024, 51(1): 143-149.
[12] YI Qiuhua, GAO Haoran, CHEN Xinqi, KONG Xiangjie. Human Mobility Pattern Prior Knowledge Based POI Recommendation [J]. Computer Science, 2023, 50(9): 139-144.
[13] LI Rongchang, ZHENG Haibin, ZHAO Wenhong, CHEN Jinyin. Data Reconstruction Attack for Vertical Graph Federated Learning [J]. Computer Science, 2023, 50(7): 332-338.
[14] JIANG Linpu, CHEN Kejia. Self-supervised Dynamic Graph Representation Learning Approach Based on Contrastive Prediction [J]. Computer Science, 2023, 50(7): 207-212.
[15] LI Fan, JIA Dongli, YAO Yumin, TU Jun. Graph Neural Network Few Shot Image Classification Network Based on Residual and Self-attention Mechanism [J]. Computer Science, 2023, 50(6A): 220500104-5.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!