计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (7): 207-212.doi: 10.11896/jsjkx.220500093

• 人工智能 • 上一篇    下一篇

基于对比预测的自监督动态图表示学习方法

蒋林浦1, 陈可佳1,2   

  1. 1 南京邮电大学计算机学院 南京 210003
    2 江苏省大数据安全与智能处理重点实验室(南京邮电大学) 南京 210023
  • 收稿日期:2022-05-11 修回日期:2022-10-09 出版日期:2023-07-15 发布日期:2023-07-05
  • 通讯作者: 陈可佳(chenkj@njupt.edu.cn)
  • 作者简介:(1020041111@njupt.edu.cn)
  • 基金资助:
    国家自然科学基金(61876091); 南京大学计算机软件新技术国家重点实验室开放课题(KFKT2022B01);南京邮电大学校级科研基金(NY221071)

Self-supervised Dynamic Graph Representation Learning Approach Based on Contrastive Prediction

JIANG Linpu1, CHEN Kejia1,2   

  1. 1 School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    2 Jiangsu Key Laboratory of Big Data Security & Intelligent Processing(Nanjing University of Posts and Telecommunications),Nanjing 210023,China
  • Received:2022-05-11 Revised:2022-10-09 Online:2023-07-15 Published:2023-07-05
  • About author:JIANG Linpu,born in 1997,postgra-duate.His main research interests include dynamic graph learning and contrast learning.CHEN Kejia,born in 1980,Ph.D,asso-ciate professor.Her main research inte-rests include complex network analysis and graph learning.
  • Supported by:
    National Natural Science Foundation of China(61876091),National Key Laboratory of New Technology of Computer Software of Nanjing University(KFKT2022B01) and Research Foundation of Nanjing University of Posts and Telecommunications(NY221071).

PDF (PC)

323

摘要： 近年来,以图对比学习为代表的图自监督学习已成为图学习领域的热点研究问题,该类学习范式不依赖于节点的标签并具有良好的泛化能力。然而,大多数图自监督学习方法采用静态图结构设计学习任务,如对比图的结构学习节点级或者图级的表示等,而未考虑图随时间的动态变化信息。为此,文中提出了一种基于对比预测的自监督动态图表示学习方法(DGCP),利用对比损失引导嵌入空间捕获对预测未来图结构最有用的信息。首先,利用图神经网络对每个时间快照图编码,得到对应的节点表示矩阵;然后,使用自回归模型预测下一时间快照图中的节点表示;最后,利用对比损失和滑动窗口机制对模型进行端到端的训练。在真实图数据集上进行实验,结果表明,DGCP在链接预测任务上的表现优于基准方法。

关键词: 动态图表示学习, 对比学习, 图神经网络, 链接预测

Abstract: In recent years,graph self-supervised learning represented by graph contrastive learning has become a hot research to-pic in the field of graph learning.This learning paradigm does not depend on node labels and has good generalization ability.However,most of the existing graph self-supervised learning methods use static graph structures to design learning tasks,such as learning node-level or graph-level representations based on structural contrast,without considering the dynamic information of graph over time.To address this problem,the paper proposes a self-supervised dynamic graph representation learning method based on contrastive prediction(DGCP),which utilizes a contrastive loss inducing the embedding space to capture the most useful information for predicting future graph structures.Firstly,each temporal snapshot graph is encoded using a graph neural network to obtain its corresponding node representation matrix.Then,an autoregressive model is used to predict node representations in the next temporal snapshot graph.Finally,the model is trained end-to-end by using the contrastive loss and sliding window me-chanism.Experimental results on real graph datasets show that DGCP outperforms baseline methods on the link prediction task.

Key words: Dynamic graph representation learning, Contrast learning, Graph neural network, Link prediction

中图分类号: 

  • TP391
[1]CHEN J,MA T,XIAO C.FastGCN:Fast Learning with Graph Convolutional Networks via Importance Sampling[C]//Procee-dings of the 6th International Conference on Learning Representations.2018.
[2]HOU H,HILDRUN K,LIU Z.The Structure of Scientific Collaboration Networks in Scientometrics [J].Scientometrics,2008,75(2):189-202.
[3]LIAO R,ZHAO Z,RAQUEL U,et al.LanczosNet:Multi-Scale Deep Graph Convolutional Networks[C]//Proceedings of the 7th International Conference on Learning Representations.2019.
[4]PETAR V,GUILLEM C,ARANTXA C,et al.Graph Attention Networks[C]//Proceedings of the 6th International Conference on Learning Representations.2018.
[5]WU F,AMAURI H S J,ZHANG T,et al.Simplifying Graph Convolutional Networks[C]//Proceedings of the 36th International Conference on Machine Learning.2019:6861-6871.
[6]THOMAS N K,MAX W.Semi-Supervised Classification withGraph Convolutional Networks[C]//Proceedings of the 5th International Conference on Learning Representations.2017.
[7]QU M,YOSHUA B,TANG J.GMNN:Graph Markov Neural Networks[C]//Proceedings of the 36th International Confe-rence on Machine Learning.2019:5241-5250.
[8]LIU Y,PAN S,JIN M,et al.Graph Self-supervised Learning:a Survey [J].arXiv:2103.00111,2021.
[9]BRYAN P,RAMI A,STEVEN S.DeepWalk:Online Learning of Social Representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2014:701-710.
[10]ADITYA G,JURE L.Node2vec:Scalable Feature Learning for Networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2016:855-864.
[11]PETAR V,WILLIAM F,WILLIAM L H.Deep Graph Infomax[C]//Proceedings of the 7th International Conference on Lear-ning Representations.2019.
[12]ZHU Y,XU Y,YU F,et al.Graph Contrastive Learning with Adaptive Augmentation[C]//Proceedings of the Web Confe-rence.2021:2069-2080.
[13]PENG Z,HUANG W,LUO M,et al.Graph RepresentationLearning via Graphical Mutual Information Maximization[C]//Proceedings of the Web Conference.2020:259-270.
[14]PETER E.Predictive Coding-I [J].IRE Transactions on Information Theory,1955,1(1):16-24.
[15]BISHNU S A,MANFRED R S.Adaptive Predictive Coding of Speech Signals [J].The Bell System Technical Journal,1970,49(8):1973-1986.
[16]GIANG H N,JOHN B L,RYAN A R,et al.Continuous-Time Dynamic Network Embeddings[C]//Proceedings of the Web Conference.2018:969-976.
[17]ZUO Y,LIU G,LIN H,et al.Embedding Temporal Network via Neighborhood Formation[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2018:2857-2866.
[18]RAKSHIT T,MEGRDAD F,PRASENJEET B,et al.DyRep:Learning Representations over Dynamic Graphs[C]//Procee-dings of the 7th International Conference on Learning Representations.2019.
[19]EMANUELE R,BEN C,FABRIZIO F,et al.Temporal GraphNetworks for Deep Learning on Dynamic Graphs [J].arXiv:2006.10637,2020.
[20]PALASH G,NITIN K,XINRAN H,et al.DynGEM:Deep Embedding Method for Dynamic Graphs [J].arXiv:1805.11273,2018.
[21]ZHOU L K,YANG Y,REN X,et al.Dynamic Network Embedding by Modeling Triadic Closure Process[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence.2018:571-578.
[22]PALASH G,SUJIT R C,ARQUIMEDES C.Dyngraph2vec:Capturing Network Dynamics Using Dynamic Graph Representation Learning [J].Knowledge Base System,2020,187:104816.1-104816.9.
[23]ARAVIND S,WU Y,GOU L,et al.DySAT:Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks[C]//Proceedings of the 13th ACM International Confe-rence on Web Search and Data Mining.2020:519-527.
[24]JING L,TIAN Y.Self-supervised Visual Feature Learning with Deep Neural Networks:a Survey [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(11):4037-4058.
[25]LIU X,ZHANG F,HOU Z,et al.Self-supervised Learning:Generative or Contrastive[J].arXiv:2006.08218,2020.
[26]AARON V D O,LI Y,ORIOLO V.Representation Learning with Contrastive Predictive Coding [J].arXiv:1807.03748,2018.
[27]HJELM R D,ALEX F,SAMUEL L,et al.Learning Deep Representations by Mutual Information Estimation and Maximization[C]//Proceedings of the 7th International Conference on Learning Representations.2019.
[28]ISHMAEL B,SAI R,ARISTIDE B,et al.MINE:Mutual Information Neural Estimation [J].arXiv:1801.04062,2018.
[29]ZHU Y,XU Y,YU F,et al.Deep Graph Contrastive Representation Learning [J].arXiv:2006.04131,2020.
[30]TIAN S,WU R,SHI L,et al.Self-supervised RepresentationLearning on Dynamic Graphs[C]//Proceedings of the 30th ACM International Conference on Information and Knowledge Management.2021:1814-1823.
[31]JONAS G,MICHAEL A,DAVID G,et al.Convolutional Sequence to Sequence Learning[C]//Proceedings of the 34th International Conference on Machine Learning.2017:1243-1252.
[32]ASHISH V,NOAM S,NIKI P,et al.Attention is All you Need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:6000-6010.
[33]YOU Y,CHEN T,WANG Z,et al.When does Self-Supervision Help Graph Convolutional Networks?[C]//Proceedings of the 37th International Conference on Machine Learning.2020:10871-10880.
[34]MARTIN A,ASHISH A,PAUL B,et al.Tensorflow:Large-scale Machine Learning on Heterogeneous Distributed Systems [J].arXiv:1603.04467,2016.
[35]DIEDERIK P K,JIMM Y B.Adam:A Method for Stochastic Optimization[C]//Proceedings of the 3rd International Confe-rence on Learning Representations.2015.
[36]HARPER F M,JOSEPH A K.The Movielens Datasets:History and Context [J].ACM Transactions on Interactive Intelligent Systems,2016,5(4):1-19.
[37]WILL H,YING Z,JURE L.Inductive Representation Learning on Large Graphs [C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:1024-1034.
[38]MARINKA Z,MONICA A,JURE L.Modeling Polypharmacy Side Effects with Graph Convolutional Networks [J].Bioinformatics,2018,34(13):457-466.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发