计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (10): 227-233.doi: 10.11896/jsjkx.230800167

• 数据库&大数据&数据科学 • 上一篇    下一篇

结构影响力及标签冲突感知的图课程学习方法

刘祖龙1, 陈可佳1,2,3   

  1. 1 南京邮电大学计算机学院 南京 210023
    2 江苏省大数据安全与智能处理重点实验室(南京邮电大学) 南京 210023
    3 南京大学软件新技术重点实验室 南京 210023
  • 收稿日期:2023-08-25 修回日期:2023-01-04 出版日期:2024-10-15 发布日期:2024-10-11
  • 通讯作者: 陈可佳(chenkj＠njupt.edu.cn)
  • 作者简介:(1021041210@njupt.edu.cn)
  • 基金资助:
    国家自然科学基金(61876091);南京大学计算机软件新技术国家重点实验室开放课题(KFKT2022B01);南京邮电大学校级科研基金(NY221071)

Structural Influence and Label Conflict Aware Based Graph Curriculum Learning Approach

LIU Zulong1, CHEN Kejia1,2,3   

  1. 1 School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China
    2 Jiangsu Key Laboratory of Big Data Security & Intelligent Processing (Nanjing University of Posts and Telecommunications),Nanjing 210023,China
    3 State Key Laboratory for Novel Software Technology at Nanjing University,Nanjing 210023,China
  • Received:2023-08-25 Revised:2023-01-04 Online:2024-10-15 Published:2024-10-11
  • About author:LIU Zulong,born in 1997,postgra-duate.His main research interests include graph representation learning and graph structure learning.
    CHEN Kejia,born in 1980,Ph.D,asso-ciate professor.Her main research in-terests include complex network analysis and graph learning.
  • Supported by:
    National Natural Science Foundation of China(61876091),Foundation of State Key Laboratory for Novel Software Technology at Nanjing University(KFKT2022B01) and Research Foundation of Nanjing University of Posts and Telecommunications(NY221071).

PDF (PC)

229

摘要： 近年来,图神经网络(GNNs)已成为图学习领域的热点研究问题。受益于消息传递机制,GNNs在各类基于图的任务上均取得了优越的性能。现有的GNNs方法大多基于图中所有节点的训练难度相同的假设,然而,节点在结构影响力和邻域标签异配性等方面具有明显的差异。为此,提出了一种结构影响力及标签冲突感知的图课程学习方法(SILC-GCL),基于节点的训练难度对GNNs模型进行课程学习。首先,设计了一种综合考虑节点的PageRank影响力值以及邻域标签冲突程度的训练难度测量器;其次,采用了一个训练调度器,用于在每个训练阶段选择训练难度合适的节点并生成一个由易到难的训练节点序列;最后在GNNs骨架模型上进行训练。在6个现实网络数据集上进行的节点分类实验均验证了SILC-GCL方法的有效性。

关键词: 图表示学习, 图神经网络, 课程学习, 节点分类

Abstract: In recent years,graph neural networks(GNNs) have emerged as a prominent research area in the field of graph lear-ning.Leveraging the message passing mechanism,GNNs have showcased remarkable performance across diverse graph-based tasks.However,most existing GNNs methods assume uniform training difficulty across all nodes,disregarding the significant va-riability in the importance and contributions of different nodes.To address this problem,this paper proposes a structural influence and label conflict aware graph curriculum learning method(SILC-GCL),which takes the training difficulty of nodes into conside-ration.To begin with,a difficulty measure is designed through both the graph structure and node label semantics,calculating the PageRank value of nodes and the label conflict degree between nodes and their neighbors.Subsequently,a training scheduler is employed to select nodes with appropriate training difficulty at each training stage and then generate a sequence of training nodes from easy to difficult.Finally,SILC-GCL is trained based on backbone GNNs models.Experimental results of node classification on six benchmark datasets verify the effectiveness of SILC-GCL.

Key words: Graph representation learning, Graph neural networks, Curriculum learning, Node classification

中图分类号: 

  • TP391
[1]WU Z,PAN S,CHEN F,et al.A comprehensive survey ongraph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(1):4-24.
[2]GOYAL P,FERRARA E.Graph embedding techniques,applica-tions,and performance:A survey[J].Knowledge-Based Systems,2018,151:78-94.
[3]KUMAR A,SINGH S S,SINGH K,et al.Link prediction techniques,applications,and performance:A survey[J].Physica A:Statistical Mechanics and Its Applications,2020,553:124289.
[4]ZHOU J,CUI G,HU S,et al.Graph neural networks:A review of methods and applications[J].AI Open,2020,1:57-81.
[5]HE D,WANG T,ZHAI L,et al.Adversarial RepresentationMechanism Learning for Network Embedding[J].IEEE Tran-sactions on Knowledge & Data Engineering,2023,35(2):1200-1213.
[6]JIN D,HUO C,LIANG C,et al.Heterogeneous graph neuralnetwork via attribute completion[C]//Proceedings of the ACM Web Conference.2021:391-400.
[7]THOMAS N K,MAX W.Semi-Supervised Classification withGraph Convolutional Networks[C]//Proceedings of the International Conference on Learning Representations.2017.
[8]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:1025-1035.
[9]BARABÁSI A L,BONABEAU E.Scale-free networks[J].Scientific American,2003,288(5):60-69.
[10]PAGE L,BRIN S,MOTWANI R,et al.The pagerank citation ranking:Bring order to the web[R].Stanford University:Technical Report,1998.
[11]CIOTTI V,BONAVENTURA M,NICOSIA V,et al.Homophily and missing links in citation networks[J].EPJ Data Science,2016,5:1-14.
[12]LIU Y,ZHENG Y,ZHANG D,et al.Towards unsuperviseddeep graph structure learning[C]//Proceedings of the ACM Web Conference.2022:1392-1403.
[13]ZHENG X,LIU Y,PAN S,et al.Graph neural networks for graphs with heterophily:A survey[J].arXiv:2202.07082,2022.
[14]ZHU J,YAN Y,ZHAO L,et al.Beyond homophily in graph neural networks:Current limitations and effective designs[J].Advances in Neural Information Processing Systems,2020,33:7793-7804.
[15]SOVIANY P,IONESCU R T,ROTA P,et al.Curriculum lear-ning:A survey[J].International Journal of Computer Vision,2022,130(6):1526-1565.
[16]LIU Z,NGUYEN T K,FANG Y.Tail-gnn:Tail-node graphneural networks[C]//Proceedings of the ACM SIGKDD Conference on Knowledge Discovery & Data Mining.2021:1109-1119.
[17]ZHENG W,HUANG E W,RAO N,et al.Cold Brew:Distilling Graph Node Representations with Incomplete or Missing Neighborhoods[C]//International Conference on Learning Representations.2021.
[18]XIA F,WANG L,TANG T,et al.Cengcn:Centralized convolutional networks with vertex imbalance for scale-free graphs[J].IEEE Transactions on Knowledge and Data Engineering,2022,35(5):4555-4569.
[19]SUN Q,LI J,YUAN H,et al.Position-aware structure learning for graph topology-imbalance by relieving under-reaching and over-squashing[C]//Proceedings of the ACM International Conference on Information & Knowledge Management.2022:1848-1857.
[20]FU X,WEI Y,SUN Q,et al.Hyperbolic Geometric Graph Representation Learning for Hierarchy-imbalance Node Classification[C]//Proceedings of the ACM Web Conference.2023:460-468.
[21]GUO S,HUANG W,ZHANG H,et al.Curriculumnet:Weakly supervised learning from large-scale web images[C]//Procee-dings of the European Conference on Computer Vision (ECCV).2018:135-150.
[22]JIANG L,MENG D,MITAMURA T,et al.Easy samples first:Self-paced reranking for zero-example multimedia search[C]//Proceedings of the ACM International Conference on Multi-media.2014:547-556.
[23]PLATANIOS E A,STRETCU O,NEUBIG G,et al.Compe-tence-based Curriculum Learning for Neural Machine Translation[C]//Proceedings of NAACL-HLT.2019:1162-1172.
[24]TAY Y,WANG S,LUU A T,et al.Simple and Effective Curri-culum Pointer-Generator Networks for Reading Comprehension over Long Narratives[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics.2019:4922-4931.
[25]LI H,WANG X,ZHU W.Curriculum Graph Machine Lear-ning:A Survey[J].arXiv:2302.02926,2023.
[26]WANG Y,WANG W,LIANG Y,et al.Curgraph:Curriculumlearning for graph classification[C]//Proceedings of the ACM Web Conference.2021:1238-1248.
[27]CHU G,WANG X,SHI C,et al.CuCo:Graph Representation with Curriculum Contrastive Learning[C]// Proceedings of the International Joint Conference on Artificial Intelligence.2021:2300-2306.
[28]WANG H,ZHOU K,ZHAO X,et al.Curriculum pre-trainingheterogeneous subgraph transformer for top-n recommendation[J].ACM Transactions on Information Systems,2023,41(1):1-28.
[29]WEI X,GONG X,ZHAN Y,et al.Clnode:Curriculum learning for node classification[C]//Proceedings of the ACM International Conference on Web Search and Data Mining.2023:670-678.
[30]LI X,WEN L,DENG Y,et al.Graph neural network with curriculum learning for imbalanced node classification[J].arXiv:2202.02529,2022.
[31]CHAWLA N V,BOWYER K W,HALL L O,et al.SMOTE:synthetic minority over-sampling technique[J].Journal of Artificial Intelligence Research,2002,16:321-357.
[32]LIU Y,YANG X,ZHOU S,et al.Hard sample aware network for contrastive deep graph clustering[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2023,37(7):8914-8922.
[33]HACOHEN G,WEINSHALL D.On the power of curriculumlearning in training deep networks[C]// Proceedings of International Conference on Machine Learning.2019:2535-2544.
[34]PLATANIOS E A,STRETCU O,NEUBIG G,et al.Compe-tence-based Curriculum Learning for Neural Machine Translation[C]//Proceedings of NAACL-HLT.2019:1162-1172.
[35]MCCALLUM A K,NIGAM K,RENNIE J,et al.Automating the construction of internet portals with machine learning[J].Information Retrieval,2000,3:127-163.
[36]GILES C L,BOLLACKER K D,LAWRENCE S.CiteSeer:An automatic citation indexing system[C]//Proceedings of the ACM Conference on Digital Libraries.1998:89-98.
[37]SEN P,NAMATA G,BILGIC M,et al.Collective classification in network data[J].AI Magazine,2008,29(3):93.
[38]HU W,FEY M,ZITNIK M,et al.Open graph benchmark:Datasets for machine learning on graphs[J].Advances in Neural Information Processing Systems,2020,33:22118-22133.
[39]PEI H,WEI B,CHANG K C C,et al.Geom-GCN:Geometric Graph Convolutional Networks[C]//International Conference on Learning Representations.2019.
[40]ROZEMBERCZKI B,ALLEN C,SARKAR R.Multi-Scale at-tributed node embedding[J].Journal of Complex Networks,2021,9(1):1-22.
[41]TANG J,SUN J,WANG C,et al.Social influence analysis in large-scale networks[C]//Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mi-ning.2009:807-816.
[42]VELIČKOVIĆ P,CUCURULL G,CASANOVA A,et al.GraphAttention Networks[C]//International Conference on Learning Representations.2018.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发