计算机科学 ›› 2024, Vol. 51 ›› Issue (11): 157-165.doi: 10.11896/jsjkx.231000209

• 计算机图形学&多媒体 • 上一篇    下一篇

基于特征插值的深度图对比聚类算法

杨希洪1, 郑群2, 章佳欣1, 王沛1, 祝恩1   

  1. 1 国防科技大学计算机学院 长沙 410073
    2 中国科学技术大学地球和空间科学学院 合肥 230001
  • 收稿日期:2023-10-30 修回日期:2024-04-05 出版日期:2024-11-15 发布日期:2024-11-06
  • 通讯作者: 祝恩(enzhu@nudt.edu.cn)
  • 作者简介:(yangxihong@nudt.edu.cn)
  • 基金资助:
    国家科技重大专项(2022ZD0209103)

Feature Interpolation Based Deep Graph Contrastive Clustering Algorithm

YANG Xihong1, ZHENG Qun2, ZHANG Jiaxin1, WANG Pei1, ZHU En1   

  1. 1 College of Computer Science and Technology,National University of Defense Technology,Changsha 410073,China
    2 School of Earth and Space Sciences,University of Science and Technology of China,Hefei 230001,China
  • Received:2023-10-30 Revised:2024-04-05 Online:2024-11-15 Published:2024-11-06
  • About author:YANG Xihong,born in 1999,Ph.D candidate.His main research interests include self supervised graph representation learning,recommendation system,deep multi-view learning,etc.
    ZHU En,born in 1976, professor,Ph.D supervisor, is a senior member of CCF(No.16689D).His main research in-terests include clustering, anomaly detection,computer vision,medical image analysis,etc.
  • Supported by:
    National Science and Technology Major Project (2022ZD0209103).

摘要: Mixup是图像领域中一种有效的数据增强方法,它通过对输入图像以及标签进行插值来合成新的样本进而扩大训练分布。然而,在图节点聚类任务中,由于图数据拓扑结构的不规则性和连通性以及无监督的场景,设计有效的插值方法成为一项具有挑战性的任务。为了解决上述问题,首先通过设计不共享参数的编码器来获取视图的嵌入特征,有效融合节点的特征和结构信息。然后将视图的嵌入特征及其对应的伪标签进行混合插值,从而将Mixup引入聚类任务中。为了确保伪标签的可靠性,设置了阈值来筛选高置信度的伪标签,并通过EMA的方式更新模型参数,使模型平稳优化的同时考虑了训练的历史信息。此外,设计了一个图对比学习模块,以保证特征在不同视图下的一致性,从而减少信息冗余,提高模型的判别能力。最终,通过在6个数据集上的大量实验证明了所提方法的有效性。

关键词: 数据增强, 图对比聚类, EMA, Mixup, 图神经网络

Abstract: Mixup is an effective data augmentation technique in the field of computer vision.It is widely used for expanding the training distribution by interpolating input images and labels to generate new samples.However,in the context of graph node clustering tasks,designing robust interpolation methods poses challenges due to the irregularity and connectivity of graph data,as well as the unsupervised nature of the problem.To address these challenges,we propose a novel approach that leverages a dedicated encoder with non-shared parameters to extract embedding features from different views of graph.This allows us to effectively integrate both the node features and structural information.We then introduce Mixup into the clustering task by performing mixed interpolation on the embedding features along with their corresponding pseudo-labels.To ensure the reliability of these pseudo-labels,we apply a threshold to filter out high-confidence predictions,while incorporating an exponential moving average(EMA) mechanism for updating model parameters and considering the historical information during training.Furthermore,we incorporate a graph contrastive learning module to enhance feature consistency across different views,reducing information redundancy and improving the discriminative power of the model.Extensive experiments on six datasets demonstrate the effectiveness of the proposed method.

Key words: Data augmentation, Graph contrastive clustering, EMA, Mixup, Graph neural network

中图分类号: 

  • TP391
[1] HUANG Y J,CHEN M,ZHENG Y,et al.Text Classification Based on Weakened Graph Convolutional Networks[J].Computer Science,2023,50(S1):220700039-5.
[2] LI F,JIA D L,YAO Y,et al.Graph Neural Network Few Shot Image Classification Network Based on Residual and Self-attention Mechanism[J].Computer Science,2023,50(S1):220500104-5.
[3] YANG Y,ZHANG F,LI T R.Aspect-based Sentiment AnalysisBased on Dual-channel Graph Convolutional Network with Sentiment Knowledge[J].Computer Science,2023,50(5):230-237.
[4] WANG Y L,ZHANG F,YU Z,et al.Aspect-level Sentiment Classification Based on Interactive Attention and Graph Convolutional Network[J].Computer Science,2023,50(4):196-203.
[5] YANG X,LIU Y,ZHOU S,et al.Cluster-guided ContrastiveGraph Clustering Network[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2023:10834-10842.
[6] XU D,CHENG W,LUO D,et al.Spatio-Temporal Attentive RNN for Node Classification in Temporal Attributed Graphs[C]//IJCAI.2019:3947-3953.
[7] XIE J,GIRSHICK R,FARHADI A.Unsupervised Deep Embedding for Clustering Analysis[C]//International Conference on Machine Learning.PMLR,2016:478-487.
[8] YANG B,FU X,SIDIROPOULOS N D,et al.Towards K-means-friendly Spaces:Simultaneous Deep Learning and Clustering[C]//International Conference on Machine Learning.PMLR,2017:3861-3870.
[9] WANG C,PAN S,LONG G,et al.Mgae:Marginalized Graph Autoencoder for Graph Clustering[C]//Proceedings of the 2017 ACM Conference on Information and Knowledge Management.2017:889-898.
[10] PAN S,HU R,FUNG S,et al.Learning Graph Embedding with Adversarial Training Methods[J].IEEE Transactions on Cybernetics,2019,50(6):2475-2487.
[11] LI X,ZHANG H,ZHANG R.Adaptive Graph Auto-encoder for General Data Clustering[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(12):9725-9732.
[12] ZHU Y,XU Y,YU F,et al.Graph Contrastive Learning with Adaptive Augmentation[C]//Proceedings of the Web Confe-rence 2021.2021:2069-2080.
[13] LEE N,LEE J,PARK C.Augmentation-free Self-supervisedLearning on Graphs[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:7372-7380.
[14] JIN W,LIU X,ZHAO X,et al.Automated Self-SupervisedLearning for Graphs[J].arXiv:2106.05470,2021.
[15] PAN E,KANG Z.Multi-view Contrastive Graph Clustering[J].Advances in Neural Information ProcessingSystems,2021,34:2148-2159.
[16] LIU Y,TU W,ZHOU S,et al.Deep Graph Clustering via Dual Correlation Reduction[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.2022:7603-7611.
[17] YANG X,HU X,ZHOU S,et al.Interpolation-based Contrastive Learning for Few-label Semi-Supervised Learning[J].IEEE Transactions on Neural Networks and Learning Systems,2024,35(2):2054-2065.
[18] GAO C,WANG X,HE X,et al.Graph Neural Networks forRecommender System[C]//Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining.2022:1623-1625.
[19] WANG C,PAN S,HU R,et al.Attributed Graph Clustering:A Deep Attentional Embedding Approach[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence.2019:3670-3676.
[20] VASWANI A,SHAZEER N,PARMAR N,et al.Attention isAll You Need[J].Advances in Neural Information Processing Systems,2017,30:1-11.
[21] BO D,WANG X,SHI C,et al.Structural Deep Clustering Network[C]//Proceedings of the Web Conference 2020.2020:1400-1410.
[22] LIU Y,YANG X,ZHOU S,et al.Hard Sample Aware Network for Contrastive Deep Graph Clustering[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2023:8914-8922.
[23] HASSANI K,KHASAHMADI A H.Contrastive Multi-viewRepresentation Learning on Graphs[C]//International Confe-rence on Machine Learning.PMLR,2020:4116-4126.
[24] ZHAO H,YANG X,WANG Z,et al.Graph Debiased Contrastive Learning with Joint Representation Clustering[C]//IJCAI.2021:3434-3440.
[25] XIA W,WANG Q,GAO Q,et al.Self-consistent ContrastiveAttributed Graph Clustering with Pseudo-label Prompt[J].IEEE Transactions on Multimedia,2023,25:6665-6677.
[26] CUI G,ZHOU J,YANG C,et al.Adaptive Graph Encoder for Attributed Graph Embedding[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.2020:976-985.
[27] WANG Y,WANG W,LIANG Y,et al.Mixup for Node and Graph Classification[C]//Proceedings of the Web Conference 2021.2021:3663-3674.
[28] VERMA V,KAWAGUCHI K,LAMB A,et al.InterpolationConsistency Training for Semi-Supervised Learning[J].Neural Networks,2022,145:90-106.
[29] HE K,FAN H,WU Y,et al.Momentum Contrast for Unsupervised Visual Representation Learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:9729-9738.
[30] PLUMMER M D,LOV’ASZ L.Matching theory[M].Elsevier,1986.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!