Computer Science ›› 2021, Vol. 48 ›› Issue (11A): 81-87.doi: 10.11896/jsjkx.210300036

• Intelligent Computing • Previous Articles     Next Articles

Deep Clustering Model Based on Fusion Variational Graph Attention Self-encoder

KANG Yan, KOU Yong-qi, XIE Si-yu, WANG Fei, ZHANG Lan, WU Zhi-wei, LI Hao   

  1. School of Software,Yunnan University,Kunming 650504,China
  • Online:2021-11-10 Published:2021-11-12
  • About author:KANG Yan,born in 1972,Ph.D,associate professor.Her main research interests include transfer learning,deep learning and integrated learning.
    LI Hao,born in 1970,Ph.D,professor. His main research interests include distributed computing,grid and cloud computing.
  • Supported by:
    National Natural Science Foundation(61762092),Key Laboratory of Software Engineering(2020SE303) and Major Scientific Research Plan of Yunnan Province(202002AB080001).

Abstract: As one of the most basic tasks in data mining and machine learning,clustering is widely used in various real-world tasks.With the development of deep learning deep clustering has become a research hotspot.Existing deep clustering algorithms are mainly from two aspects of node representation learning or structural representation learning.Less work considers fusing these two kinds of information at the same time to complete representation learning.This paper proposes a deep clustering model FVGTAEDC (Deep Clustering Model Based on Fusion Varitional Graph Attention Self-encoder),this model joints the autoencoderand the variational graph attention autoencoder for clustering.In the model,the autoencoder integrates the variational graph attention autoencoder from the network to learn (low-order and high-order) structural representations,and then the feature representation is learned from the original data.While the two modules are trained,in order to adapt to the clustering task,self-supervised clustering training for the autoencoder module is integrated with the representation features of the node and the structure information.Comprehensive clustering loss,autoencoder reconstruction data loss,and variational graph attention autoencoder reconstruction adjacency matrix loss,the relative entropy loss of the posterior probability distribution and the prior probability distribution.The method can effectively aggregate the attributes of nodes and the structure of the network,while optimizing the assignment of cluster labels and learning the representation features suitable for clustering.Comprehensive experiments prove that the method is better than the current advanced deep clustering method on 5 real data.

Key words: Deep clustering, Representation learning, Self encoder, Self-supervised clustering, Variational graph attention self-encoder

CLC Number: 

  • TP181
[1]YANG B,LIU D Y,LIU J M,et al.Complex network clustering method[J].Journal of Software,2009,20(1):54-66.
[2]AHARTIGAN J,WONG M A.Algorithm AS 136:A k-means clustering algorithm.Journal of the Royal Statistical Society[J].Series C (Applied Statistics),1979,28(1):100-108.
[3]CHANG J L,WANG L F,MENG G F,et al.Deep AdaptiveImage Clustering[C]//IEEE International Conference on Computer Vision.2017:5880-5888.
[4]CAGGARWAL C,ZHAI C X.A survey of text clustering algorithms[C]//Mining Text Data.Springer.2012:77-128.
[5]SHANG J W,WANG C K,XIN X,et al.Community discovery algorithm based on deep sparse autoencoder[J].Journal of Software,2017,28(3):648-662.
[6]ARTHUR D,ASSILVITSKII S V.k-means++:The advantages of careful seeding[C]//SODA.2007:1027-1035.
[7]ESTER M,KRIEGEL H P,SANDER J,et al.A density-based algorithm for discovering clusters in large spatial databases with noise[C]//KDD.1996:226-231.
[8]POUYANFAR S,SADIQ S,YAN Y L,et al.A Survey on Deep Learning:Algorithms,Techniques,and Applications[C]//ACM Computing Surveys.2019:1-36.
[9]TIAN F,GAO B,CUI Q,et al.Learning Deep Representations for Graph Clustering[C]//AAAI.2014:1293-1299.
[10]XIE J Y,GIRSHICK R,FARHADI A.Unsupervised deep embedding for clustering analysis[C]//ICML.2016:478-487.
[11]GUO X F,GAO L,LIU X W,et al.Improved deep embeddedclustering with local structure preservation[C]//IJCAI.2017:1753-1759.
[12]PENG X,XIAO S J,FENG J S,et al.Deep subspace clustering with sparsity prior[C]//IJCAI.2016:101-115.
[13]JIANG Z X,ZHENG Y,TAN H C,et al.Variational deep embedding:An unsupervised and generative approach to clustering[C]//IJCAI.2017:4305-4324.
[14]NKIPF T,WELLING M.Semi-supervised classification withgraph convolutional networks[C]//ICLR.2017:1-14.
[15]NKIPF T,WELLING M.Variational graph auto-encoders[J].NIPS,2016,21(11):1-3.
[16]WANG C,PAN S R,HU R Q,et al.Attributed Graph Clustering:A Deep Attentional Embedding Approach[C]//IJCAI.Marina del Rey CA USA:Association for the Advancement of Artificial Intelligence (AAAI),2019:3670-3676.
[17]LI X L,ZHANG H Y,ZHANG R.Embedding Graph Auto-Encoder with Joint Clustering via Adjacency Sharing[C]//WWW.2020:1-11.
[18]WANG C,PAN S R,LONG G D,et al.MGAE:MarginalizedGraph Autoencoder for Graph Clustering[C]//ACM on Conference on Information and Knowledge Management.2017:889-898.
[19]ZHANG X T,LIU H,LI Q M,et al.Attributed Graph Clustering via Adaptive Graph Convolution[C]//IJCAI.2019:4327-4333.
[20]BO D Y,WANG X,SHI C,et al.Structural Deep Clustering Network[C]//WWW.2020:1-11.
[21]SUN J G,LIU J,ZHAO L Y.Research on clustering algorithm[J].Journal of Software,2008,19(1):48-61.
[22]JAIN A K,DUBES R C.Algorithms for clustering data[J].Technometrics,1988,32(2):227-229.
[23]REYNOLDS D A.Gaussian mixture models[C]//Encyclopedia of Biometrics.2015:1-23.
[24]JOHNSON S C.Hierarchical clustering schemes[J].Psy-
chometrika,1967,32(3):241-254.
[25]NG A Y,JORDAN M I,WEISS Y.On spectral clustering:Analy-sis and an algorithm[C]//Advances in Neural Information Processing Systems.2002:849-856.
[26]HINTON G E,SALAKHUTDINOV R R.Reducing the dimensional ity of data with neural networks[J].Science,2006,7(28):504-507.
[27]HINTON G E.Learning multiple layers of representation[J].Science,2007,7(4):428-434.
[28]RAMACHANDRAN P,ZOPH B,LE Q V.Searching For Activation Functions[C]//ICLR.2018:1-13.
[29]CASANOVA A,ROMERO A,LIO P,et al.Graph AttentionNetworks[C]//IJCAI.2018:1-12.
[30]CHEPURI S P,LEUS G.Subsampling For Graph Power Spectrum Estimation[C]//IEEE SAM.2016:1250-1263.
[31]VAN DER MAATEN L,HINTON G.Visualizing data using t-sne[J].Journal of Machine Learning Research,2008,9(Nov):2579-2605.
[32]DENKER J,GARDNER W R,GRAF H,et al.Neural Network Recognizer for Hand-Written Zip Code Digits[C]//NIPS.1988:323-331.
[33]STISEN A,BLUNCK H,BHATTACHARYA S,et al.Smart devices are different:Assessing and mitigatingmobile sensing heterogeneities for activity recognition[C]//SenSys.ACM,2015:127-140.
[1] SONG Jie, LIANG Mei-yu, XUE Zhe, DU Jun-ping, KOU Fei-fei. Scientific Paper Heterogeneous Graph Node Representation Learning Method Based onUnsupervised Clustering Level [J]. Computer Science, 2022, 49(9): 64-69.
[2] HUANG Li, ZHU Yan, LI Chun-ping. Author’s Academic Behavior Prediction Based on Heterogeneous Network Representation Learning [J]. Computer Science, 2022, 49(9): 76-82.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] LI Zong-min, ZHANG Yu-peng, LIU Yu-jie, LI Hua. Deformable Graph Convolutional Networks Based Point Cloud Representation Learning [J]. Computer Science, 2022, 49(8): 273-278.
[5] HUANG Pu, DU Xu-ran, SHEN Yang-yang, YANG Zhang-jing. Face Recognition Based on Locality Regularized Double Linear Reconstruction Representation [J]. Computer Science, 2022, 49(6A): 407-411.
[6] JIANG Zong-li, FAN Ke, ZHANG Jin-li. Generative Adversarial Network and Meta-path Based Heterogeneous Network Representation Learning [J]. Computer Science, 2022, 49(1): 133-139.
[7] WANG Ying-li, JIANG Cong-cong, FENG Xiao-nian, QIAN Tie-yun. Time Aware Point-of-interest Recommendation [J]. Computer Science, 2021, 48(9): 43-49.
[8] ZHAO Jin-long, ZHAO Zhong-ying. Recommendation Algorithm Based on Heterogeneous Information Network Embedding and Attention Neural Network [J]. Computer Science, 2021, 48(8): 72-79.
[9] YE Hong-liang, ZHU Wan-ning, HONG Lei. Music Style Transfer Method with Human Voice Based on CQT and Mel-spectrum [J]. Computer Science, 2021, 48(6A): 326-330.
[10] YANG Ru-han, DAI Yi-ru, WANG Jian, DONG Jin. Humans-Cyber-Physical Ontology Fusion of Industry Based on Representation Learning [J]. Computer Science, 2021, 48(5): 190-196.
[11] QIAN Sheng-sheng, ZHANG Tian-zhu, XU Chang-sheng. Survey of Multimedia Social Events Analysis [J]. Computer Science, 2021, 48(3): 97-112.
[12] WANG Xing , KANG Zhao. Smooth Representation-based Semi-supervised Classification [J]. Computer Science, 2021, 48(3): 124-129.
[13] LI Xin-chao, LI Pei-feng, ZHU Qiao-ming. Directed Network Representation Method Based on Hierarchical Structure Information [J]. Computer Science, 2021, 48(2): 100-104.
[14] WANG Xue-cen, ZHANG Yu, LIU Ying-jie, YU Ge. Evaluation of Quality of Interaction in Online Learning Based on Representation Learning [J]. Computer Science, 2021, 48(2): 207-211.
[15] FU Kun, ZHAO Xiao-meng, FU Zi-tong, GAO Jin-hui, MA Hao-ran. Deep Network Representation Learning Method on Incomplete Information Networks [J]. Computer Science, 2021, 48(12): 212-218.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!