计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (6): 282-298.doi: 10.11896/jsjkx.230400005

• 人工智能 • 上一篇    下一篇

图神经网络研究综述

侯磊1, 刘金环1, 于旭2, 杜军威1   

  1. 1 青岛科技大学数据科学学院 山东 青岛 266061
    2 中国石油大学(华东)计算机科学与技术学院 山东 青岛 266580
  • 收稿日期:2023-04-03 修回日期:2023-09-30 出版日期:2024-06-15 发布日期:2024-06-05
  • 通讯作者: 刘金环(liujinhuan.sdu@gmail.com)
  • 作者简介:(houlei_fly@163.com)
  • 基金资助:
    国家自然科学基金(62202253,62172249);山东省自然科学基金(ZR2021QF074,ZR2021MF092)

Review of Graph Neural Networks

HOU Lei1, LIU Jinhuan1, YU Xu2, DU Junwei1   

  1. 1 School of Data Science,Qingdao University of Science and Technology,Qingdao,Shandong 266061,China
    2 School of Computer Science and Technology,China University of Petroleum (East China),Qingdao,Shandong 266580,China
  • Received:2023-04-03 Revised:2023-09-30 Online:2024-06-15 Published:2024-06-05
  • About author:HOU Lei,born in 1998,postgraduate,is a member of CCF(No.Q7342G).His main research interests include recommendation system,graph neural networks and information retrieval.
    LIU Jinhuan,born in 1989,Ph.D,is a member of CCF(No.I1628M).Her mainresearch interests include machine learning,recommendation system,and information retrieval.
  • Supported by:
    National Natural Science Foundation of China(62202253,62172249) and National Natural Science Foundation of Shandong Province,China(ZR2021QF074,ZR2021MF092).

PDF (PC)

692

摘要： 随着人工智能的快速发展,深度学习已经在图像、文本和语音等可在欧氏空间表示的数据中取得了巨大成功,但却一直无法很好地应用于非欧氏空间。近年来,图神经网络在非欧几里得空间中展现出了强大的表示学习能力,并广泛应用于推荐系统、自然语言处理以及机器视觉等众多领域。图神经网络模型基于信息的传播机制,具体地,图中的目标节点通过聚合邻居节点的信息来更新自身的嵌入表示。利用图神经网络,可将众多现实问题(如社交网络、知识图谱和药物化学成分等)抽象成图网络,借助图中的连接边,对不同节点之间的依赖关系进行合理建模。鉴于此,对图神经网络进行了系统综述,首先介绍了图结构数据方面的基础知识,然后对图游走算法和不同类型的图神经网络模型进行了系统梳理。进一步地,详细阐述了当前图神经网络的通用框架和应用领域,最后对图神经网络的未来进行了总结与展望。

关键词: 图结构数据, 图游走算法, 图卷积神经网络, 图注意力网络, 图残差网络, 图递归网络

Abstract: With the rapid development of artificial intelligence,deep learning has achieved great success in data that can be represented in Euclidean spaces,such as images,text,and speech.However,it has been difficult to apply deep learning to non-Eucli-dean spaces.In recent years,with the emergence of graph neural networks,it has demonstrated powerful representation learning abilities in non-Euclidean spaces and has been widely applied in various fields such as recommendation systems,natural language processing,and computer vision.The graph neural network model is based on the mechanism of information propagation.Specifi-cally,the target node in the graph updates its embedding representation by aggregating the information of neighboring nodes.With graph neural networks,many real-world problems(such as social networks,knowledge graphs,and drug chemical compositions) can be abstracted into graph networks and the dependence relationships between different nodes can be modeled reasonably using the connecting edges in the graph.Therefore,this paper systematically reviews graph neural networks,introduces the basic knowledge of graph-structured data,and systematically reviews graph walk algorithms and different types of graph neural network models.Furthermore,it also details the current general framework and application areas of graph neural networks,and concludes with a summary and outlook on future research in graph neural networks.

Key words: Graph-structure data, Graph walk algorithm, Graph convolutional networks, Graph attention networks, Graph residual networks, Graph recurrent networks

中图分类号: 

  • TP183
[1]GU J,WANG Z,KUEN J,et al.Recent advances in convolu-tional neural networks[J].Pattern Recognition,2018,77:354-377.
[2]MEDSKER L R,JAIN L C.Recurrent neural networks[J].Design and Applications,2001,5:64-67.
[3]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial networks[J].Communications of the ACM,2020,63(11):139-144.
[4]GORI M,MONFARDINI G,SCARSELL F.A new model for learning in graph domains[C]//2005 IEEE International Joint Conference on Neural Networks.New York:IEEE,2005:729-734.
[5]SCARSELLI F,GORI M,TSOI A C,et al.The graph neural network model[J].IEEE Transactions on Neural Networks,2008,20(1):61-80.
[6]NIEPERT M,AHMED M,KUTZKOV K.Learning convolu-tional neural networks for graphs[C]//International Conference on Machine Learning.New York:ACM,2016:2014-2023.
[7]GAO H,WANG Z,JI S.Large-scale learnable graph convolu-tional networks[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mi-ning.New York:ACM,2018:1416-1424.
[8]ATWOOD J,TOWSIEV D.Diffusion-convolutional neural networks[J].Advances in Neural Information Processing Systems,2016,9:2001-2009.
[9]PEROZZI B,AL-RFOU R,SKIENA S.Deepwalk:Online lear-ning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Disco-very and Data Mining.New York:ACM,2014:701-710.
[10]GROVER A,LESKOVEC J.node2vec:Scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mi-ning.New York:ACM,2016:855-864.
[11]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[J].arXiv:1310.4546,2013.
[12]BERG R,KIPF T N,WELLING M.Graph convolutional matrix completion[J].arXiv:1706.02263,2017.
[13]VELICKOVIC P,CUCURULL G,Casanova A,et al.Graph attention networks[J].arXiv:1710.10903,2018.
[14]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:1025-1035.
[15]HE X,DENG K,WANG X,et al.Lightgcn:Simplifying andpowering graph convolution network for recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York:ACM,2020:639-648.
[16]FAN W,MA Y,LI Q,et al.Graph neural networks for social recommendation[C]//The world Wide Web Conference.New York:ACM,2019:417-426.
[17]LIU F,CHENG Z,ZHU L,et al.An attribute-aware attentive GCN model for attribute missing in recommendation[J].IEEE Transactions on Knowledge and Data Engineering,2020,34(9):4077-4088.
[18]WANG X,HE X,CAO Y,et al.Kgat:Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.New York:ACM,2019:950-958.
[19]FAN S,ZHU J,HAN X,et al.Metapath-guided heterogeneous graph neural network for intent recommendation[C]//Procee-dings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.New York:ACM,2019:2478-2486.
[20]VASHISHTH S,BHANDARI M,YADAV P,et al.Incorporating syntactic and semantic information in word embeddings using graph convolutional networks[J].arXiv:1809.04283,2018.
[21]WANG K,SHEN W,YANG Y,et al.Relational graph attention network for aspect-based sentiment analysis[J].arXiv:2004.12362,2020.
[22]LIU Z,XIONG C,SUN M,et al.Fine-grained fact verificationwith kernel graph attention network[J].arXiv:1910.09796,2019.
[23]WANG D,LIU P,ZHENG Y,et al.Heterogeneous graph neural networks for extractive document summarization[J].arXiv:2004.12393,2020.
[24]CHEN Z M,WEI X S,WANG P,et al.Multi-label image recognition with graph convolutional networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2019:5177-5186.
[25]GAO J,ZHANG T,XU C.I know the relationships:Zero-shot action recognition via two-stream graph convolutional networks and knowledge graphs[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.Menlo Park:AAAI,2019,33(1):8303-8311.
[26]HAN K,WANG Y,GUO J,et al.Vision GNN:An Image is Worth Graph of Nodes[J].arXiv:2206.00272,2022.
[27]ZHANG Z,SHI Y,YUAN C,et al.Object relational graph with teacher-recommended learning for video captioning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2020:13278-13288.
[28]MOHAMED A,QIAN K,ELHOSEINY M,et al.Social-stgcnn:A social spatio-temporal graph convolutional neural network for human trajectory prediction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2020:14424-14432.
[29]HAN K,LAKSHMINARAYANNA B,LIU J.Reliable graph neural networks for drug discovery under distributional shift[J].arXiv:2111.12951,2021.
[30]ZHAO H,WANG Y,DUAN J,et al.Multivariate time-seriesanomaly detection via graph attention network[C]//2020 IEEE International Conference on Data Mining(ICDM).New York:IEEE,2020:841-850.
[31]DENG A,HOOI B.Graph neural network-based anomaly detection in multivariate time series[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI,2021,35(5):4027-4035.
[32]CUI Z,LI Z,WU S,et al.Dressing as a whole:Outfit compatibility learning based on node-wisegraph neural networks[C]//The World Wide Web Conference.New York:ACM,2019:307-317.
[33]TONG Z,LIANG Y,SUN C,et al.Directed graph convolutional network[J].arXiv:2004.13970,2020.
[34]KAMPFFMEVER M,CHEN Y,LIANG X,et al.Rethinkingknowledge graph propagation for zero-shot learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2019:11487-11496.
[35]DUAN D,LI Y,JIN Y,et al.Community mining on dynamic weighted directed graphs[C]//Proceedings of the 1st ACM International Workshop on Complex Networks Meet Information &Knowledge Management.New York:ACM,2009:11-18.
[36]ZHANG Y,XIONG Y,KONG X,et al.Deep collective classification in heterogeneous information networks[C]//Proceedings of the 2018 World Wide Web Conference.New York:ACM,2018:399-408.
[37]LI Y,YU R,SHAHABI C,et al.Diffusion convolutional recurrent neural network:Data-driven traffic forecasting[J].arXiv:1707.01926,2017.
[38]YAN S,XIONG Y,LIN D.Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI,2018.
[39]HUANG Y,XU H,DUAN Z,et al.Modeling complex spatial patterns with temporal features via heterogenous graph embedding networks[J].arXiv:2008.08617,2020.
[40]BECK D,HAFFARI G,COHN T.Graph-to-sequence learning using gated graph neural networks[J].arXiv:1806.09835,2018.
[41]LI Y,TARLOW D,BROCKSCHMIDT M,et al.Gated graphsequence neural networks[J].arXiv:1511.05493,2015.
[42]ABDI H,WILLIAMA L J.Principal component analysis[J].WileyInterdisciplinary Reviews:Computational Statistics,2010,2(4):433-459.
[43]BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation[J].Journal of machine Learning Research,2003,3(Jan):993-1022.
[44]CARDOSO J F,COMON P.Independent component analysis,a survey of some algebraic methods[C]//1996 IEEE International Symposium on Circuits and Systems,Circuits and Systems Connecting the World(ISCAS 96).New York:IEEE,1996:93-96.
[45]TANG J,QU M,WANG M,et al.Line:Large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web.New York:ACM,2015:1067-1077.
[46]ERUNA J,ZAREMBA W,SZLAM A,et al.Spectral networks and deep locally connected networks on graphs[C]//2nd International Conference on Learning Representations.2014.
[47]MA S,LIU J W,ZUO X.Survey on Graph Neural Network[J].Journal of Computer Research and Development,2022,59(1):47-80.
[48]DEFFERRARD M,BRESSON X,VANDERGHEYNST P.Convolutional neural networks on graphs with fast localized spectral filtering[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems.2016:3844-3852.
[49]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[50]LI R,WANG S,ZHU F,et al.Adaptive graph convolutional neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI,2018.
[51]XU B,SHEN H,CAO Q,et al.Graph wavelet neural network[J].arXiv:1904.07785,2019.
[52]MICHELI A.Neural network for graphs:A contextual con-structive approach[J].IEEE Transactions on Neural Networks,2009,20(3):498-511.
[53]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv:1409.0473,2014.
[54]GEHRING J,AULI M,GRANGIER D,et al.A convolutionalencoder model for neural machine translation[J].arXiv:1611.02344,2016.
[55]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[56]CHENG J,DONG L,LAPATA M.Long short-term memory-networks for machine reading[J].arXiv:1601.06733,2016.
[57]ZHANG J,SHI X,XIE J,et al.Gaan:Gated attention networks for learning on large and spatiotemporal graphs[J].arXiv:1803.07294,2018.
[58]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2016:770-778.
[59]LI G,MULLER M,THABET A,et al.Deepgcns:Can gcns go as deep as cnns?[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.New York:IEEE,2019:9267-9276.
[60]HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely con-nected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:4700-4708.
[61]YU F,KOLTUN V.Multi-scale context aggregation by dilated convolutions[J].arXiv:1511.07122,2015.
[62]ZHENG Y,GAO C,CHEN L,et al.Dgcn:Diversified recommendation with graph convolutional networks[C]//Proceedings of the Web Conference 2021.New York:ACM,2021:401-412.
[63]LIANG X,SHEN X,FENG J,et al.Semantic object parsingwith graph lstm[C]//Computer Vision－ECCV 2016:14th European Conference,Amsterdam,The Netherlands,Part I 14.Amsterdam:Springer,2016:125-143.
[64]WU Z,PAN S,CHEN F,et al.A comprehensive survey ongraph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(1):4-24.
[65]GILMER J,SCHOENHOLZ S S,RILEY P F,et al.Neural message passing for quantum chemistry[C]//International Confe-rence on Machine Learning.PMLR,2017:1263-1272.
[66]WANG X,GIRSHICK R,GUPTA A,et al.Non-local neuralnetworks[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.2018:7794-7803.
[67]BATTAGLIA P W,HAMRICK J B,BAPST V,et al.Relational inductive biases,deep learning,and graph networks[J].arXiv:1806.01261,2018.
[68]DUVENAUD D K,MACLAURIN D,IPARRAGUIRRE J,et al.Convolutional networks on graphs for learning molecular fingerprints[C]//Proceedings of the 28st International Confe-rence on Neural Information Processing Systems December.2015:2224-2232.
[69]SCHUTT K T,ARBABZADAH F,CHMIELA S,et al.Quantum-chemical insights from deep tensor neural networks[J].Nature Communications,2017,8(1):1-8.
[70]BUADES A,COLL B,MOREL J M.A non-local algorithm for image denoising[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR’05).New York:IEEE,2005:60-65.
[71]BATTAGLIA P,PASCANU R,LAI M,et al.Interaction networks for learning about objects,relations and physics[C]//Proceedings of the 28st International Conference on Neural Information Processing Systems December.2016:4509-4517.
[72]TOMASI C,MANDUCHI R.Bilateral filtering for gray and co-lor images[C]//Sixth international Conference on Computer Vision.New York:IEEE,1998:839-846.
[73]HAMRICK J B,ALLEN K R,BAPST V,et al.Relational inductive bias for physical construction in humans and machines[J].arXiv:1806.01203,2018.
[74]ZHANG M,CHEN Y.Inductive matrix completion based ongraph neural networks[J].arXiv:1904.12058,2019.
[75]YING R,HE R,CHEN K,et al.Graph convolutional neural networks for web-scale recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.New York:ACM,2018:974-983.
[76]ZHANG C,LI Q,SONG D.Aspect-based sentiment classification with aspect-specific graph convolutional networks[J].arXiv:1909.03477,2019.
[77]ZHANG N,DENG S,SUN Z,et al.Long-tail relation extraction via knowledge graph embeddings and graph convolution networks[J].arXiv:1903.01306,2019.
[78]SAXENA A,TRIPATHI A,TALUKDAR P.Improving multi-hop question answering over knowledge graphs using knowledge base embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:4498-4507.
[79]PENG H,LI J,HE Y,et al.Large-scale hierarchical text classification with recursively regularized deep graph-cnn[C]//Proceedings of the 2018 World Wide Web Conference.2018:1063-1072.
[80]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019,33(1):7370-7377.
[81]WANG X,YE Y,GUPTA A.Zero-shot recognition via semantic embeddings and knowledge graphs[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2018:6857-6866.
[82]GARCIA V,BRUNA J.Few-shot learning with graph neuralnetworks[J].arXiv:1711.04043,2017.
[83]SHI W,RAJKUMAR R.Point-gnn:Graph neural network for 3d object detection in a point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE,2020:1711-1719.
[84]TENEY D,AMDERSON P,HE X,et al.Tips and tricks for visual question answering:Learnings from the 2017 challenge[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:4223-4232.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发