Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (6): 282-298.doi: 10.11896/jsjkx.230400005

• Artificial Intelligence • Previous Articles     Next Articles

Review of Graph Neural Networks

HOU Lei1, LIU Jinhuan1, YU Xu2, DU Junwei1   

  1. 1 School of Data Science,Qingdao University of Science and Technology,Qingdao,Shandong 266061,China
    2 School of Computer Science and Technology,China University of Petroleum (East China),Qingdao,Shandong 266580,China
  • Received:2023-04-03 Revised:2023-09-30 Online:2024-06-15 Published:2024-06-05
  • About author:HOU Lei,born in 1998,postgraduate,is a member of CCF(No.Q7342G).His main research interests include recommendation system,graph neural networks and information retrieval.
    LIU Jinhuan,born in 1989,Ph.D,is a member of CCF(No.I1628M).Her mainresearch interests include machine learning,recommendation system,and information retrieval.
  • Supported by:
    National Natural Science Foundation of China(62202253,62172249) and National Natural Science Foundation of Shandong Province,China(ZR2021QF074,ZR2021MF092).

PDF (PC)

684

Abstract: With the rapid development of artificial intelligence,deep learning has achieved great success in data that can be represented in Euclidean spaces,such as images,text,and speech.However,it has been difficult to apply deep learning to non-Eucli-dean spaces.In recent years,with the emergence of graph neural networks,it has demonstrated powerful representation learning abilities in non-Euclidean spaces and has been widely applied in various fields such as recommendation systems,natural language processing,and computer vision.The graph neural network model is based on the mechanism of information propagation.Specifi-cally,the target node in the graph updates its embedding representation by aggregating the information of neighboring nodes.With graph neural networks,many real-world problems(such as social networks,knowledge graphs,and drug chemical compositions) can be abstracted into graph networks and the dependence relationships between different nodes can be modeled reasonably using the connecting edges in the graph.Therefore,this paper systematically reviews graph neural networks,introduces the basic knowledge of graph-structured data,and systematically reviews graph walk algorithms and different types of graph neural network models.Furthermore,it also details the current general framework and application areas of graph neural networks,and concludes with a summary and outlook on future research in graph neural networks.

Key words: Graph-structure data, Graph walk algorithm, Graph convolutional networks, Graph attention networks, Graph residual networks, Graph recurrent networks

CLC Number: 

  • TP183
[1]GU J,WANG Z,KUEN J,et al.Recent advances in convolu-tional neural networks[J].Pattern Recognition,2018,77:354-377.
[2]MEDSKER L R,JAIN L C.Recurrent neural networks[J].Design and Applications,2001,5:64-67.
[3]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial networks[J].Communications of the ACM,2020,63(11):139-144.
[4]GORI M,MONFARDINI G,SCARSELL F.A new model for learning in graph domains[C]//2005 IEEE International Joint Conference on Neural Networks.New York:IEEE,2005:729-734.
[5]SCARSELLI F,GORI M,TSOI A C,et al.The graph neural network model[J].IEEE Transactions on Neural Networks,2008,20(1):61-80.
[6]NIEPERT M,AHMED M,KUTZKOV K.Learning convolu-tional neural networks for graphs[C]//International Conference on Machine Learning.New York:ACM,2016:2014-2023.
[7]GAO H,WANG Z,JI S.Large-scale learnable graph convolu-tional networks[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mi-ning.New York:ACM,2018:1416-1424.
[8]ATWOOD J,TOWSIEV D.Diffusion-convolutional neural networks[J].Advances in Neural Information Processing Systems,2016,9:2001-2009.
[9]PEROZZI B,AL-RFOU R,SKIENA S.Deepwalk:Online lear-ning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Disco-very and Data Mining.New York:ACM,2014:701-710.
[10]GROVER A,LESKOVEC J.node2vec:Scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mi-ning.New York:ACM,2016:855-864.
[11]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[J].arXiv:1310.4546,2013.
[12]BERG R,KIPF T N,WELLING M.Graph convolutional matrix completion[J].arXiv:1706.02263,2017.
[13]VELICKOVIC P,CUCURULL G,Casanova A,et al.Graph attention networks[J].arXiv:1710.10903,2018.
[14]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:1025-1035.
[15]HE X,DENG K,WANG X,et al.Lightgcn:Simplifying andpowering graph convolution network for recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York:ACM,2020:639-648.
[16]FAN W,MA Y,LI Q,et al.Graph neural networks for social recommendation[C]//The world Wide Web Conference.New York:ACM,2019:417-426.
[17]LIU F,CHENG Z,ZHU L,et al.An attribute-aware attentive GCN model for attribute missing in recommendation[J].IEEE Transactions on Knowledge and Data Engineering,2020,34(9):4077-4088.
[18]WANG X,HE X,CAO Y,et al.Kgat:Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.New York:ACM,2019:950-958.
[19]FAN S,ZHU J,HAN X,et al.Metapath-guided heterogeneous graph neural network for intent recommendation[C]//Procee-dings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.New York:ACM,2019:2478-2486.
[20]VASHISHTH S,BHANDARI M,YADAV P,et al.Incorporating syntactic and semantic information in word embeddings using graph convolutional networks[J].arXiv:1809.04283,2018.
[21]WANG K,SHEN W,YANG Y,et al.Relational graph attention network for aspect-based sentiment analysis[J].arXiv:2004.12362,2020.
[22]LIU Z,XIONG C,SUN M,et al.Fine-grained fact verificationwith kernel graph attention network[J].arXiv:1910.09796,2019.
[23]WANG D,LIU P,ZHENG Y,et al.Heterogeneous graph neural networks for extractive document summarization[J].arXiv:2004.12393,2020.
[24]CHEN Z M,WEI X S,WANG P,et al.Multi-label image recognition with graph convolutional networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2019:5177-5186.
[25]GAO J,ZHANG T,XU C.I know the relationships:Zero-shot action recognition via two-stream graph convolutional networks and knowledge graphs[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.Menlo Park:AAAI,2019,33(1):8303-8311.
[26]HAN K,WANG Y,GUO J,et al.Vision GNN:An Image is Worth Graph of Nodes[J].arXiv:2206.00272,2022.
[27]ZHANG Z,SHI Y,YUAN C,et al.Object relational graph with teacher-recommended learning for video captioning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2020:13278-13288.
[28]MOHAMED A,QIAN K,ELHOSEINY M,et al.Social-stgcnn:A social spatio-temporal graph convolutional neural network for human trajectory prediction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2020:14424-14432.
[29]HAN K,LAKSHMINARAYANNA B,LIU J.Reliable graph neural networks for drug discovery under distributional shift[J].arXiv:2111.12951,2021.
[30]ZHAO H,WANG Y,DUAN J,et al.Multivariate time-seriesanomaly detection via graph attention network[C]//2020 IEEE International Conference on Data Mining(ICDM).New York:IEEE,2020:841-850.
[31]DENG A,HOOI B.Graph neural network-based anomaly detection in multivariate time series[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI,2021,35(5):4027-4035.
[32]CUI Z,LI Z,WU S,et al.Dressing as a whole:Outfit compatibility learning based on node-wisegraph neural networks[C]//The World Wide Web Conference.New York:ACM,2019:307-317.
[33]TONG Z,LIANG Y,SUN C,et al.Directed graph convolutional network[J].arXiv:2004.13970,2020.
[34]KAMPFFMEVER M,CHEN Y,LIANG X,et al.Rethinkingknowledge graph propagation for zero-shot learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2019:11487-11496.
[35]DUAN D,LI Y,JIN Y,et al.Community mining on dynamic weighted directed graphs[C]//Proceedings of the 1st ACM International Workshop on Complex Networks Meet Information &Knowledge Management.New York:ACM,2009:11-18.
[36]ZHANG Y,XIONG Y,KONG X,et al.Deep collective classification in heterogeneous information networks[C]//Proceedings of the 2018 World Wide Web Conference.New York:ACM,2018:399-408.
[37]LI Y,YU R,SHAHABI C,et al.Diffusion convolutional recurrent neural network:Data-driven traffic forecasting[J].arXiv:1707.01926,2017.
[38]YAN S,XIONG Y,LIN D.Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI,2018.
[39]HUANG Y,XU H,DUAN Z,et al.Modeling complex spatial patterns with temporal features via heterogenous graph embedding networks[J].arXiv:2008.08617,2020.
[40]BECK D,HAFFARI G,COHN T.Graph-to-sequence learning using gated graph neural networks[J].arXiv:1806.09835,2018.
[41]LI Y,TARLOW D,BROCKSCHMIDT M,et al.Gated graphsequence neural networks[J].arXiv:1511.05493,2015.
[42]ABDI H,WILLIAMA L J.Principal component analysis[J].WileyInterdisciplinary Reviews:Computational Statistics,2010,2(4):433-459.
[43]BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation[J].Journal of machine Learning Research,2003,3(Jan):993-1022.
[44]CARDOSO J F,COMON P.Independent component analysis,a survey of some algebraic methods[C]//1996 IEEE International Symposium on Circuits and Systems,Circuits and Systems Connecting the World(ISCAS 96).New York:IEEE,1996:93-96.
[45]TANG J,QU M,WANG M,et al.Line:Large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web.New York:ACM,2015:1067-1077.
[46]ERUNA J,ZAREMBA W,SZLAM A,et al.Spectral networks and deep locally connected networks on graphs[C]//2nd International Conference on Learning Representations.2014.
[47]MA S,LIU J W,ZUO X.Survey on Graph Neural Network[J].Journal of Computer Research and Development,2022,59(1):47-80.
[48]DEFFERRARD M,BRESSON X,VANDERGHEYNST P.Convolutional neural networks on graphs with fast localized spectral filtering[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems.2016:3844-3852.
[49]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[50]LI R,WANG S,ZHU F,et al.Adaptive graph convolutional neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI,2018.
[51]XU B,SHEN H,CAO Q,et al.Graph wavelet neural network[J].arXiv:1904.07785,2019.
[52]MICHELI A.Neural network for graphs:A contextual con-structive approach[J].IEEE Transactions on Neural Networks,2009,20(3):498-511.
[53]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[54]GEHRING J,AULI M,GRANGIER D,et al.A convolutionalencoder model for neural machine translation[J].arXiv:1611.02344,2016.
[55]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[56]CHENG J,DONG L,LAPATA M.Long short-term memory-networks for machine reading[J].arXiv:1601.06733,2016.
[57]ZHANG J,SHI X,XIE J,et al.Gaan:Gated attention networks for learning on large and spatiotemporal graphs[J].arXiv:1803.07294,2018.
[58]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2016:770-778.
[59]LI G,MULLER M,THABET A,et al.Deepgcns:Can gcns go as deep as cnns?[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.New York:IEEE,2019:9267-9276.
[60]HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely con-nected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:4700-4708.
[61]YU F,KOLTUN V.Multi-scale context aggregation by dilated convolutions[J].arXiv:1511.07122,2015.
[62]ZHENG Y,GAO C,CHEN L,et al.Dgcn:Diversified recommendation with graph convolutional networks[C]//Proceedings of the Web Conference 2021.New York:ACM,2021:401-412.
[63]LIANG X,SHEN X,FENG J,et al.Semantic object parsingwith graph lstm[C]//Computer Vision－ECCV 2016:14th European Conference,Amsterdam,The Netherlands,Part I 14.Amsterdam:Springer,2016:125-143.
[64]WU Z,PAN S,CHEN F,et al.A comprehensive survey ongraph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(1):4-24.
[65]GILMER J,SCHOENHOLZ S S,RILEY P F,et al.Neural message passing for quantum chemistry[C]//International Confe-rence on Machine Learning.PMLR,2017:1263-1272.
[66]WANG X,GIRSHICK R,GUPTA A,et al.Non-local neuralnetworks[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.2018:7794-7803.
[67]BATTAGLIA P W,HAMRICK J B,BAPST V,et al.Relational inductive biases,deep learning,and graph networks[J].arXiv:1806.01261,2018.
[68]DUVENAUD D K,MACLAURIN D,IPARRAGUIRRE J,et al.Convolutional networks on graphs for learning molecular fingerprints[C]//Proceedings of the 28st International Confe-rence on Neural Information Processing Systems December.2015:2224-2232.
[69]SCHUTT K T,ARBABZADAH F,CHMIELA S,et al.Quantum-chemical insights from deep tensor neural networks[J].Nature Communications,2017,8(1):1-8.
[70]BUADES A,COLL B,MOREL J M.A non-local algorithm for image denoising[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR’05).New York:IEEE,2005:60-65.
[71]BATTAGLIA P,PASCANU R,LAI M,et al.Interaction networks for learning about objects,relations and physics[C]//Proceedings of the 28st International Conference on Neural Information Processing Systems December.2016:4509-4517.
[72]TOMASI C,MANDUCHI R.Bilateral filtering for gray and co-lor images[C]//Sixth international Conference on Computer Vision.New York:IEEE,1998:839-846.
[73]HAMRICK J B,ALLEN K R,BAPST V,et al.Relational inductive bias for physical construction in humans and machines[J].arXiv:1806.01203,2018.
[74]ZHANG M,CHEN Y.Inductive matrix completion based ongraph neural networks[J].arXiv:1904.12058,2019.
[75]YING R,HE R,CHEN K,et al.Graph convolutional neural networks for web-scale recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.New York:ACM,2018:974-983.
[76]ZHANG C,LI Q,SONG D.Aspect-based sentiment classification with aspect-specific graph convolutional networks[J].arXiv:1909.03477,2019.
[77]ZHANG N,DENG S,SUN Z,et al.Long-tail relation extraction via knowledge graph embeddings and graph convolution networks[J].arXiv:1903.01306,2019.
[78]SAXENA A,TRIPATHI A,TALUKDAR P.Improving multi-hop question answering over knowledge graphs using knowledge base embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:4498-4507.
[79]PENG H,LI J,HE Y,et al.Large-scale hierarchical text classification with recursively regularized deep graph-cnn[C]//Proceedings of the 2018 World Wide Web Conference.2018:1063-1072.
[80]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019,33(1):7370-7377.
[81]WANG X,YE Y,GUPTA A.Zero-shot recognition via semantic embeddings and knowledge graphs[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2018:6857-6866.
[82]GARCIA V,BRUNA J.Few-shot learning with graph neuralnetworks[J].arXiv:1711.04043,2017.
[83]SHI W,RAJKUMAR R.Point-gnn:Graph neural network for 3d object detection in a point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2020:1711-1719.
[84]TENEY D,AMDERSON P,HE X,et al.Tips and tricks for visual question answering:Learnings from the 2017 challenge[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:4223-4232.
[1] ZHANG Mingdao, ZHOU Xin, WU Xiaohong, QING Linbo, HE Xiaohai. Unified Fake News Detection Based on Semantic Expansion and HDGCN [J]. Computer Science, 2024, 51(4): 299-306.
[2] ZHANG Tao, CHENG Yifei, SUN Xinxu. Graph Attention Networks Based on Causal Inference [J]. Computer Science, 2023, 50(6A): 220600230-9.
[3] YANG Ying, ZHANG Fan, LI Tianrui. Aspect-based Sentiment Analysis Based on Dual-channel Graph Convolutional Network with Sentiment Knowledge [J]. Computer Science, 2023, 50(5): 230-237.
[4] LI Shuai, XU Bin, HAN Yike, LIAO Tongxin. SS-GCN:Aspect-based Sentiment Analysis Model with Affective Enhancement and Syntactic Enhancement [J]. Computer Science, 2023, 50(3): 3-11.
[5] QIN Mingfei, FU Guohong. Multi-level Semantic Structure Enhanced Emotional Cause Span Extraction in Conversations [J]. Computer Science, 2023, 50(12): 236-245.
[6] ZHANG Longji, ZHAO Hui. Aspect-level Sentiment Analysis Integrating Syntactic Distance and Aspect-attention [J]. Computer Science, 2023, 50(12): 262-269.
[7] DENG Ruhan, ZHANG Qinghua, HUANG Shuaishuai, GAO Man. Novel Graph Convolutional Network Based on Multi-granularity Feature Fusion for Aspect-basedSentiment Analysis [J]. Computer Science, 2023, 50(10): 80-87.
[8] ZHENG Cheng, MEI Liang, ZHAO Yiyan, ZHANG Suhang. Text Classification Method Based on Bidirectional Attention and Gated Graph Convolutional Networks [J]. Computer Science, 2023, 50(1): 221-228.
[9] ZHOU Hai-yu, ZHANG Dao-qiang. Multi-site Hyper-graph Convolutional Neural Networks and Application [J]. Computer Science, 2022, 49(3): 129-133.
[10] PAN Zhi-hao, ZENG Bi, LIAO Wen-xiong, WEI Peng-fei, WEN Song. Interactive Attention Graph Convolutional Networks for Aspect-based Sentiment Classification [J]. Computer Science, 2022, 49(3): 294-300.
[11] ZHANG Bin, LIU Chang-hong, ZENG Sheng, JIE An-quan. Speech-driven Personal Style Gesture Generation Method Based on Spatio-Temporal GraphConvolutional Networks [J]. Computer Science, 2022, 49(11A): 210900094-5.
[12] ZENG Wei-liang, CHEN Yi-hao, YAO Ruo-yu, LIAO Rui-xiang, SUN Wei-jun. Application of Spatial-Temporal Graph Attention Networks in Trajectory Prediction for Vehicles at Intersections [J]. Computer Science, 2021, 48(6A): 334-341.
[13] JIANG Zong-li, LI Miao-miao, ZHANG Jin-li. Graph Convolution of Fusion Meta-path Based Heterogeneous Network Representation Learning [J]. Computer Science, 2020, 47(7): 231-235.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.