Computer Science ›› 2023, Vol. 50 ›› Issue (7): 207-212.doi: 10.11896/jsjkx.220500093

• Artificial Intelligence • Previous Articles     Next Articles

Self-supervised Dynamic Graph Representation Learning Approach Based on Contrastive Prediction

JIANG Linpu1, CHEN Kejia1,2   

  1. 1 School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    2 Jiangsu Key Laboratory of Big Data Security & Intelligent Processing(Nanjing University of Posts and Telecommunications),Nanjing 210023,China
  • Received:2022-05-11 Revised:2022-10-09 Online:2023-07-15 Published:2023-07-05
  • About author:JIANG Linpu,born in 1997,postgra-duate.His main research interests include dynamic graph learning and contrast learning.CHEN Kejia,born in 1980,Ph.D,asso-ciate professor.Her main research inte-rests include complex network analysis and graph learning.
  • Supported by:
    National Natural Science Foundation of China(61876091),National Key Laboratory of New Technology of Computer Software of Nanjing University(KFKT2022B01) and Research Foundation of Nanjing University of Posts and Telecommunications(NY221071).

Abstract: In recent years,graph self-supervised learning represented by graph contrastive learning has become a hot research to-pic in the field of graph learning.This learning paradigm does not depend on node labels and has good generalization ability.However,most of the existing graph self-supervised learning methods use static graph structures to design learning tasks,such as learning node-level or graph-level representations based on structural contrast,without considering the dynamic information of graph over time.To address this problem,the paper proposes a self-supervised dynamic graph representation learning method based on contrastive prediction(DGCP),which utilizes a contrastive loss inducing the embedding space to capture the most useful information for predicting future graph structures.Firstly,each temporal snapshot graph is encoded using a graph neural network to obtain its corresponding node representation matrix.Then,an autoregressive model is used to predict node representations in the next temporal snapshot graph.Finally,the model is trained end-to-end by using the contrastive loss and sliding window me-chanism.Experimental results on real graph datasets show that DGCP outperforms baseline methods on the link prediction task.

Key words: Dynamic graph representation learning, Contrast learning, Graph neural network, Link prediction

CLC Number: 

  • TP391
[1]CHEN J,MA T,XIAO C.FastGCN:Fast Learning with Graph Convolutional Networks via Importance Sampling[C]//Procee-dings of the 6th International Conference on Learning Representations.2018.
[2]HOU H,HILDRUN K,LIU Z.The Structure of Scientific Collaboration Networks in Scientometrics [J].Scientometrics,2008,75(2):189-202.
[3]LIAO R,ZHAO Z,RAQUEL U,et al.LanczosNet:Multi-Scale Deep Graph Convolutional Networks[C]//Proceedings of the 7th International Conference on Learning Representations.2019.
[4]PETAR V,GUILLEM C,ARANTXA C,et al.Graph Attention Networks[C]//Proceedings of the 6th International Conference on Learning Representations.2018.
[5]WU F,AMAURI H S J,ZHANG T,et al.Simplifying Graph Convolutional Networks[C]//Proceedings of the 36th International Conference on Machine Learning.2019:6861-6871.
[6]THOMAS N K,MAX W.Semi-Supervised Classification withGraph Convolutional Networks[C]//Proceedings of the 5th International Conference on Learning Representations.2017.
[7]QU M,YOSHUA B,TANG J.GMNN:Graph Markov Neural Networks[C]//Proceedings of the 36th International Confe-rence on Machine Learning.2019:5241-5250.
[8]LIU Y,PAN S,JIN M,et al.Graph Self-supervised Learning:a Survey [J].arXiv:2103.00111,2021.
[9]BRYAN P,RAMI A,STEVEN S.DeepWalk:Online Learning of Social Representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2014:701-710.
[10]ADITYA G,JURE L.Node2vec:Scalable Feature Learning for Networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2016:855-864.
[11]PETAR V,WILLIAM F,WILLIAM L H.Deep Graph Infomax[C]//Proceedings of the 7th International Conference on Lear-ning Representations.2019.
[12]ZHU Y,XU Y,YU F,et al.Graph Contrastive Learning with Adaptive Augmentation[C]//Proceedings of the Web Confe-rence.2021:2069-2080.
[13]PENG Z,HUANG W,LUO M,et al.Graph RepresentationLearning via Graphical Mutual Information Maximization[C]//Proceedings of the Web Conference.2020:259-270.
[14]PETER E.Predictive Coding-I [J].IRE Transactions on Information Theory,1955,1(1):16-24.
[15]BISHNU S A,MANFRED R S.Adaptive Predictive Coding of Speech Signals [J].The Bell System Technical Journal,1970,49(8):1973-1986.
[16]GIANG H N,JOHN B L,RYAN A R,et al.Continuous-Time Dynamic Network Embeddings[C]//Proceedings of the Web Conference.2018:969-976.
[17]ZUO Y,LIU G,LIN H,et al.Embedding Temporal Network via Neighborhood Formation[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2018:2857-2866.
[18]RAKSHIT T,MEGRDAD F,PRASENJEET B,et al.DyRep:Learning Representations over Dynamic Graphs[C]//Procee-dings of the 7th International Conference on Learning Representations.2019.
[19]EMANUELE R,BEN C,FABRIZIO F,et al.Temporal GraphNetworks for Deep Learning on Dynamic Graphs [J].arXiv:2006.10637,2020.
[20]PALASH G,NITIN K,XINRAN H,et al.DynGEM:Deep Embedding Method for Dynamic Graphs [J].arXiv:1805.11273,2018.
[21]ZHOU L K,YANG Y,REN X,et al.Dynamic Network Embedding by Modeling Triadic Closure Process[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence.2018:571-578.
[22]PALASH G,SUJIT R C,ARQUIMEDES C.Dyngraph2vec:Capturing Network Dynamics Using Dynamic Graph Representation Learning [J].Knowledge Base System,2020,187:104816.1-104816.9.
[23]ARAVIND S,WU Y,GOU L,et al.DySAT:Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks[C]//Proceedings of the 13th ACM International Confe-rence on Web Search and Data Mining.2020:519-527.
[24]JING L,TIAN Y.Self-supervised Visual Feature Learning with Deep Neural Networks:a Survey [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(11):4037-4058.
[25]LIU X,ZHANG F,HOU Z,et al.Self-supervised Learning:Generative or Contrastive[J].arXiv:2006.08218,2020.
[26]AARON V D O,LI Y,ORIOLO V.Representation Learning with Contrastive Predictive Coding [J].arXiv:1807.03748,2018.
[27]HJELM R D,ALEX F,SAMUEL L,et al.Learning Deep Representations by Mutual Information Estimation and Maximization[C]//Proceedings of the 7th International Conference on Learning Representations.2019.
[28]ISHMAEL B,SAI R,ARISTIDE B,et al.MINE:Mutual Information Neural Estimation [J].arXiv:1801.04062,2018.
[29]ZHU Y,XU Y,YU F,et al.Deep Graph Contrastive Representation Learning [J].arXiv:2006.04131,2020.
[30]TIAN S,WU R,SHI L,et al.Self-supervised RepresentationLearning on Dynamic Graphs[C]//Proceedings of the 30th ACM International Conference on Information and Knowledge Management.2021:1814-1823.
[31]JONAS G,MICHAEL A,DAVID G,et al.Convolutional Sequence to Sequence Learning[C]//Proceedings of the 34th International Conference on Machine Learning.2017:1243-1252.
[32]ASHISH V,NOAM S,NIKI P,et al.Attention is All you Need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:6000-6010.
[33]YOU Y,CHEN T,WANG Z,et al.When does Self-Supervision Help Graph Convolutional Networks?[C]//Proceedings of the 37th International Conference on Machine Learning.2020:10871-10880.
[34]MARTIN A,ASHISH A,PAUL B,et al.Tensorflow:Large-scale Machine Learning on Heterogeneous Distributed Systems [J].arXiv:1603.04467,2016.
[35]DIEDERIK P K,JIMM Y B.Adam:A Method for Stochastic Optimization[C]//Proceedings of the 3rd International Confe-rence on Learning Representations.2015.
[36]HARPER F M,JOSEPH A K.The Movielens Datasets:History and Context [J].ACM Transactions on Interactive Intelligent Systems,2016,5(4):1-19.
[37]WILL H,YING Z,JURE L.Inductive Representation Learning on Large Graphs [C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:1024-1034.
[38]MARINKA Z,MONICA A,JURE L.Modeling Polypharmacy Side Effects with Graph Convolutional Networks [J].Bioinformatics,2018,34(13):457-466.
[1] LI Rongchang, ZHENG Haibin, ZHAO Wenhong, CHEN Jinyin. Data Reconstruction Attack for Vertical Graph Federated Learning [J]. Computer Science, 2023, 50(7): 332-338.
[2] LI Fan, JIA Dongli, YAO Yumin, TU Jun. Graph Neural Network Few Shot Image Classification Network Based on Residual and Self-attention Mechanism [J]. Computer Science, 2023, 50(6A): 220500104-5.
[3] GU Shouke, CHEN Wen. Function Level Code Vulnerability Detection Method of Graph Neural Network Based on Extended AST [J]. Computer Science, 2023, 50(6): 283-290.
[4] WANG Huiyan, YU Minghe, YU Ge. Deep Learning-based Heterogeneous Information Network Representation:A Survey [J]. Computer Science, 2023, 50(5): 103-114.
[5] SHAO Yunfei, SONG You, WANG Baohui. Study on Degree of Node Based Personalized Propagation of Neural Predictions forSocial Networks [J]. Computer Science, 2023, 50(4): 16-21.
[6] LI Shujing, HUANG Zengfeng. Mixed-curve for Link Completion of Multi-relational Heterogeneous Knowledge Graphs [J]. Computer Science, 2023, 50(4): 172-180.
[7] DONG Chengyu, LYU Mingqi, CHEN Tieming, ZHU Tiantian. Heterogeneous Provenance Graph Learning Model Based APT Detection [J]. Computer Science, 2023, 50(4): 359-368.
[8] WANG Jingbin, LAI Xiaolian, LIN Xinyu, YANG Xinyi. Context-aware Temporal Knowledge Graph Completion Based on Relation Constraints [J]. Computer Science, 2023, 50(3): 23-33.
[9] CHEN Fuqiang, KOU Jiamin, SU Limin, LI Ke. Multi-information Optimized Entity Alignment Model Based on Graph Neural Network [J]. Computer Science, 2023, 50(3): 34-41.
[10] YU Jian, ZHAO Mankun, GAO Jie, WANG Congyuan, LI Yarong, ZHANG Wenbin. Study on Graph Neural Networks Social Recommendation Based on High-order and Temporal Features [J]. Computer Science, 2023, 50(3): 49-64.
[11] ZHANG Qi, YU Shuangyuan, YIN Hongfeng, XU Baomin. Neural Collaborative Filtering for Social Recommendation Algorithm Based on Graph Attention [J]. Computer Science, 2023, 50(2): 115-122.
[12] HAO Jingyu, WEN Jingxuan, LIU Huafeng, JING Liping, YU Jian. Deep Disentangled Collaborative Filtering with Graph Global Information [J]. Computer Science, 2023, 50(1): 41-51.
[13] GU Xizhi, SHAO Yingxia. Fast Computation Graph Simplification via Influence-based Pruning for Graph Neural Network [J]. Computer Science, 2023, 50(1): 52-58.
[14] PU Jinyao, BU Lingmei, LU Yongmei, YE Ziming, CHEN Li, YU Zhonghua. Utilizing Heterogeneous Graph Neural Network to Extract Emotion-Cause Pairs Effectively [J]. Computer Science, 2023, 50(1): 205-212.
[15] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!