Computer Science ›› 2024, Vol. 51 ›› Issue (10): 227-233.doi: 10.11896/jsjkx.230800167

• Database & Big Data & Data Science • Previous Articles     Next Articles

Structural Influence and Label Conflict Aware Based Graph Curriculum Learning Approach

LIU Zulong1, CHEN Kejia1,2,3   

  1. 1 School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China
    2 Jiangsu Key Laboratory of Big Data Security & Intelligent Processing (Nanjing University of Posts and Telecommunications),Nanjing 210023,China
    3 State Key Laboratory for Novel Software Technology at Nanjing University,Nanjing 210023,China
  • Received:2023-08-25 Revised:2023-01-04 Online:2024-10-15 Published:2024-10-11
  • About author:LIU Zulong,born in 1997,postgra-duate.His main research interests include graph representation learning and graph structure learning.
    CHEN Kejia,born in 1980,Ph.D,asso-ciate professor.Her main research in-terests include complex network analysis and graph learning.
  • Supported by:
    National Natural Science Foundation of China(61876091),Foundation of State Key Laboratory for Novel Software Technology at Nanjing University(KFKT2022B01) and Research Foundation of Nanjing University of Posts and Telecommunications(NY221071).

Abstract: In recent years,graph neural networks(GNNs) have emerged as a prominent research area in the field of graph lear-ning.Leveraging the message passing mechanism,GNNs have showcased remarkable performance across diverse graph-based tasks.However,most existing GNNs methods assume uniform training difficulty across all nodes,disregarding the significant va-riability in the importance and contributions of different nodes.To address this problem,this paper proposes a structural influence and label conflict aware graph curriculum learning method(SILC-GCL),which takes the training difficulty of nodes into conside-ration.To begin with,a difficulty measure is designed through both the graph structure and node label semantics,calculating the PageRank value of nodes and the label conflict degree between nodes and their neighbors.Subsequently,a training scheduler is employed to select nodes with appropriate training difficulty at each training stage and then generate a sequence of training nodes from easy to difficult.Finally,SILC-GCL is trained based on backbone GNNs models.Experimental results of node classification on six benchmark datasets verify the effectiveness of SILC-GCL.

Key words: Graph representation learning, Graph neural networks, Curriculum learning, Node classification

CLC Number: 

  • TP391
[1]WU Z,PAN S,CHEN F,et al.A comprehensive survey ongraph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(1):4-24.
[2]GOYAL P,FERRARA E.Graph embedding techniques,applica-tions,and performance:A survey[J].Knowledge-Based Systems,2018,151:78-94.
[3]KUMAR A,SINGH S S,SINGH K,et al.Link prediction techniques,applications,and performance:A survey[J].Physica A:Statistical Mechanics and Its Applications,2020,553:124289.
[4]ZHOU J,CUI G,HU S,et al.Graph neural networks:A review of methods and applications[J].AI Open,2020,1:57-81.
[5]HE D,WANG T,ZHAI L,et al.Adversarial RepresentationMechanism Learning for Network Embedding[J].IEEE Tran-sactions on Knowledge & Data Engineering,2023,35(2):1200-1213.
[6]JIN D,HUO C,LIANG C,et al.Heterogeneous graph neuralnetwork via attribute completion[C]//Proceedings of the ACM Web Conference.2021:391-400.
[7]THOMAS N K,MAX W.Semi-Supervised Classification withGraph Convolutional Networks[C]//Proceedings of the International Conference on Learning Representations.2017.
[8]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:1025-1035.
[9]BARABÁSI A L,BONABEAU E.Scale-free networks[J].Scientific American,2003,288(5):60-69.
[10]PAGE L,BRIN S,MOTWANI R,et al.The pagerank citation ranking:Bring order to the web[R].Stanford University:Technical Report,1998.
[11]CIOTTI V,BONAVENTURA M,NICOSIA V,et al.Homophily and missing links in citation networks[J].EPJ Data Science,2016,5:1-14.
[12]LIU Y,ZHENG Y,ZHANG D,et al.Towards unsuperviseddeep graph structure learning[C]//Proceedings of the ACM Web Conference.2022:1392-1403.
[13]ZHENG X,LIU Y,PAN S,et al.Graph neural networks for graphs with heterophily:A survey[J].arXiv:2202.07082,2022.
[14]ZHU J,YAN Y,ZHAO L,et al.Beyond homophily in graph neural networks:Current limitations and effective designs[J].Advances in Neural Information Processing Systems,2020,33:7793-7804.
[15]SOVIANY P,IONESCU R T,ROTA P,et al.Curriculum lear-ning:A survey[J].International Journal of Computer Vision,2022,130(6):1526-1565.
[16]LIU Z,NGUYEN T K,FANG Y.Tail-gnn:Tail-node graphneural networks[C]//Proceedings of the ACM SIGKDD Conference on Knowledge Discovery & Data Mining.2021:1109-1119.
[17]ZHENG W,HUANG E W,RAO N,et al.Cold Brew:Distilling Graph Node Representations with Incomplete or Missing Neighborhoods[C]//International Conference on Learning Representations.2021.
[18]XIA F,WANG L,TANG T,et al.Cengcn:Centralized convolutional networks with vertex imbalance for scale-free graphs[J].IEEE Transactions on Knowledge and Data Engineering,2022,35(5):4555-4569.
[19]SUN Q,LI J,YUAN H,et al.Position-aware structure learning for graph topology-imbalance by relieving under-reaching and over-squashing[C]//Proceedings of the ACM International Conference on Information & Knowledge Management.2022:1848-1857.
[20]FU X,WEI Y,SUN Q,et al.Hyperbolic Geometric Graph Representation Learning for Hierarchy-imbalance Node Classification[C]//Proceedings of the ACM Web Conference.2023:460-468.
[21]GUO S,HUANG W,ZHANG H,et al.Curriculumnet:Weakly supervised learning from large-scale web images[C]//Procee-dings of the European Conference on Computer Vision (ECCV).2018:135-150.
[22]JIANG L,MENG D,MITAMURA T,et al.Easy samples first:Self-paced reranking for zero-example multimedia search[C]//Proceedings of the ACM International Conference on Multi-media.2014:547-556.
[23]PLATANIOS E A,STRETCU O,NEUBIG G,et al.Compe-tence-based Curriculum Learning for Neural Machine Translation[C]//Proceedings of NAACL-HLT.2019:1162-1172.
[24]TAY Y,WANG S,LUU A T,et al.Simple and Effective Curri-culum Pointer-Generator Networks for Reading Comprehension over Long Narratives[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics.2019:4922-4931.
[25]LI H,WANG X,ZHU W.Curriculum Graph Machine Lear-ning:A Survey[J].arXiv:2302.02926,2023.
[26]WANG Y,WANG W,LIANG Y,et al.Curgraph:Curriculumlearning for graph classification[C]//Proceedings of the ACM Web Conference.2021:1238-1248.
[27]CHU G,WANG X,SHI C,et al.CuCo:Graph Representation with Curriculum Contrastive Learning[C]// Proceedings of the International Joint Conference on Artificial Intelligence.2021:2300-2306.
[28]WANG H,ZHOU K,ZHAO X,et al.Curriculum pre-trainingheterogeneous subgraph transformer for top-n recommendation[J].ACM Transactions on Information Systems,2023,41(1):1-28.
[29]WEI X,GONG X,ZHAN Y,et al.Clnode:Curriculum learning for node classification[C]//Proceedings of the ACM International Conference on Web Search and Data Mining.2023:670-678.
[30]LI X,WEN L,DENG Y,et al.Graph neural network with curriculum learning for imbalanced node classification[J].arXiv:2202.02529,2022.
[31]CHAWLA N V,BOWYER K W,HALL L O,et al.SMOTE:synthetic minority over-sampling technique[J].Journal of Artificial Intelligence Research,2002,16:321-357.
[32]LIU Y,YANG X,ZHOU S,et al.Hard sample aware network for contrastive deep graph clustering[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2023,37(7):8914-8922.
[33]HACOHEN G,WEINSHALL D.On the power of curriculumlearning in training deep networks[C]// Proceedings of International Conference on Machine Learning.2019:2535-2544.
[34]PLATANIOS E A,STRETCU O,NEUBIG G,et al.Compe-tence-based Curriculum Learning for Neural Machine Translation[C]//Proceedings of NAACL-HLT.2019:1162-1172.
[35]MCCALLUM A K,NIGAM K,RENNIE J,et al.Automating the construction of internet portals with machine learning[J].Information Retrieval,2000,3:127-163.
[36]GILES C L,BOLLACKER K D,LAWRENCE S.CiteSeer:An automatic citation indexing system[C]//Proceedings of the ACM Conference on Digital Libraries.1998:89-98.
[37]SEN P,NAMATA G,BILGIC M,et al.Collective classification in network data[J].AI Magazine,2008,29(3):93.
[38]HU W,FEY M,ZITNIK M,et al.Open graph benchmark:Datasets for machine learning on graphs[J].Advances in Neural Information Processing Systems,2020,33:22118-22133.
[39]PEI H,WEI B,CHANG K C C,et al.Geom-GCN:Geometric Graph Convolutional Networks[C]//International Conference on Learning Representations.2019.
[40]ROZEMBERCZKI B,ALLEN C,SARKAR R.Multi-Scale at-tributed node embedding[J].Journal of Complex Networks,2021,9(1):1-22.
[41]TANG J,SUN J,WANG C,et al.Social influence analysis in large-scale networks[C]//Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mi-ning.2009:807-816.
[42]VELIČKOVIĆ P,CUCURULL G,CASANOVA A,et al.GraphAttention Networks[C]//International Conference on Learning Representations.2018.
[1] YUAN Lining, FENG Wengang, LIU Zhao. Multi-channel Graph Convolutional Networks Enhanced by Label Propagation Algorithm [J]. Computer Science, 2024, 51(8): 304-312.
[2] HU Haibo, YANG Dan, NIE Tiezheng, KOU Yue. Graph Contrastive Learning Incorporating Multi-influence and Preference for Social Recommendation [J]. Computer Science, 2024, 51(7): 146-155.
[3] WEI Ziang, PENG Jian, HUANG Feihu, JU Shenggen. Text Classification Method Based on Multi Graph Convolution and Hierarchical Pooling [J]. Computer Science, 2024, 51(7): 303-309.
[4] PENG Bo, LI Yaodong, GONG Xianfu, LI Hao. Method for Entity Relation Extraction Based on Heterogeneous Graph Neural Networks and TextSemantic Enhancement [J]. Computer Science, 2024, 51(6A): 230700071-5.
[5] LIU Wei, SONG You, ZHUO Peiyan, WU Weiqiang, LIAN Xin. Study on Kcore-GCN Anti-fraud Algorithm Fusing Multi-source Graph Features [J]. Computer Science, 2024, 51(6A): 230600040-7.
[6] LU Min, YUAN Ziting. Graph Contrast Learning Based Multi-graph Neural Network for Session-based RecommendationMethod [J]. Computer Science, 2024, 51(5): 54-61.
[7] ZHANG Liying, SUN Haihang, SUN Yufa , SHI Bingbo. Review of Node Classification Methods Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(4): 95-105.
[8] ZHANG Tao, LIAO Bin, YU Jiong, LI Ming, SUN Ruina. Benchmarking and Analysis for Graph Neural Network Node Classification Task [J]. Computer Science, 2024, 51(4): 132-150.
[9] ZHENG Cheng, SHI Jingwei, WEI Suhua, CHENG Jiaming. Dual Feature Adaptive Fusion Network Based on Dependency Type Pruning for Aspect-basedSentiment Analysis [J]. Computer Science, 2024, 51(3): 205-213.
[10] XU Tianyue, LIU Xianhui, ZHAO Weidong. Knowledge Graph and User Interest Based Recommendation Algorithm [J]. Computer Science, 2024, 51(2): 55-62.
[11] LIAO Bin, ZHANG Tao, YU Jiong, LI Min. NLGAE:A Graph Autoencoder Model Based on Improved Network Structure and Loss Functionfor Node Classification Task [J]. Computer Science, 2024, 51(10): 234-246.
[12] LUO Ruiqing, ZENG Kun, ZHANG Xinjing. Curriculum Learning Framework Based on Reinforcement Learning in Sparse HeterogeneousMulti-agent Environments [J]. Computer Science, 2024, 51(1): 301-309.
[13] GUO Yuxing, YAO Kaixuan, WANG Zhiqiang, WEN Liangliang, LIANG Jiye. Black-box Graph Adversarial Attacks Based on Topology and Feature Fusion [J]. Computer Science, 2024, 51(1): 355-362.
[14] JIANG Linpu, CHEN Kejia. Self-supervised Dynamic Graph Representation Learning Approach Based on Contrastive Prediction [J]. Computer Science, 2023, 50(7): 207-212.
[15] SHAO Yunfei, SONG You, WANG Baohui. Study on Degree of Node Based Personalized Propagation of Neural Predictions forSocial Networks [J]. Computer Science, 2023, 50(4): 16-21.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!