Computer Science ›› 2021, Vol. 48 ›› Issue (6A): 220-225.doi: 10.11896/jsjkx.210100167

• Big Data & Data Science • Previous Articles     Next Articles

Landmark-based Spectral Clustering by Joint Spectral Embedding and Spectral Rotation

LI Peng, LIU Li-jun, HUANG Yong-dong   

  1. School of Science,Dalian Minzu University,Dalian,Liaoning 116600,China
  • Online:2021-06-10 Published:2021-06-17
  • About author:LI Peng,master student.His main research interests include machine lear-ning and optimization method.
    LIU Li-jun,Ph.D,associate professor.His main research interests include neural networks,machine learning and optimization method.
  • Supported by:
    National Natural Science Foundation of China(61002039,61572018,11761001).

Abstract: Classical spectral clustering algorithms consist of two separate stages.One is spectral embedding,computing eigenvalue decomposition of a Laplacian matrix to obtain a relaxed continuous indication matrix.The other is post processing,applying k-means or spectral rotation to round the real matrix into the binary cluster indicator matrix.Such a separate scheme is not guaranteed to achieve jointly optimal result because of the loss of useful information.Meanwhile,there are difficulties of low clustering precision,high storage cost for the similarity matrix and high computational complexity for the eigenvalue decomposition of Laplacian matrix.The existing joint model adopts an orthonormal real matrix to approximate the orthogonal but nonorthonormal cluster indicator matrix.The error of approximating a nonorthonormal matrix is inevitably large.To overcome the drawback,we propose replacing the nonorthonormal cluster indicator matrix with an improved orthonormal cluster indicator matrix.The proposed method is capable of obtaining better performance because it is easy to minimize the difference between two orthonormal matrices.Furthermore,a novel landmark-based joint spectral embedding and spectral rotation algorithm is proposed based on the sparse representation by landmark points,which greatly solves the effective computation of spectral clustering for large scale dataset.Experimental results on benchmark datasets demonstrate the effectiveness of the proposed method.

Key words: Joint spectral clustering, Landmark representation, Spectral clustering, Spectral embedding, Spectral rotation

CLC Number: 

  • TP181
[1] JAIN A K,MURTY M N,FLYNN P J.Data clustering:a review[J].ACM Computing Surveys,1999,31(3):264-323.
[2] MACQUEEN J.Some methods for classification and analysis of multivariate observations[J].Berkeley Symposium on Mathematical Statistics and Probability,1967,1(5):281-297.
[3] FU G Y.Optimization methods for fuzzy clustering[J].Fuzzy Sets and Systems,1998,93(3):301-309.
[4] KAUFMAN L,ROUSSEEUW P J.Finding Groups in Data:An Introduction to Cluster Analysis[J].John Wiley & Sons,New York,USA,1990.
[5] LUXBURG U V.A tutorial on spectral clustering[J].Statistics and Computing,2007,17(4):395-416.
[6] JIN J G.Review of Clustering Method[J].Computer Science,2014,41(S2):288-293.
[7] LI P Q,LI Y D,DENG X L,et al.Spectral Clustering Algorithm Based on SimRank Score[J].Computer Science,2018,45(S2):458-461.
[8] LI X Y,GUO L J.Constructing affinity matrix in spectral clustering based on neighbor propagation[J].Neurocomputing,2012,97:125-130.
[9] TAO X M,WANG R T,CHANG R,et al.Low Density Separation Density Sensitive Distance-based Spectral Clustering Algorithm[J].Acta Automatica Sinica,2020,46(7):1479-1495.
[10] CHEN J F,ZHANG M,HE Q.Heuristically Determining Cluster Numbers Based NJW Spectral Clustering Algorithm[J].Computer Science,2018,45(S2):474-479.
[11] ZHANG R G,YAO X L,ZHAO J,et al.Manifold Spectral Clustering Image Segmentation Algorithm Based on Local Geometry Features[J].Pattern Recognition and Artificial Intelligence,2020,33(4):313-324.
[12] CHARLESS F,SERGE B,FAN C,et al.Spectral grouping using the Nyström method[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(2):214-225.
[13] QIU Y F,LIU C.Spectral Clustering Algorithm Based onWeighted Ensemble Nyström Sampling[J].Pattern Recognition and Artificial Intelligence,2019,32(5):420-428.
[14] CHEN X L,CAI D.Large scale spectral clustering with landmark-based representation[C]//AAAI Conference on Artificial Intelligence.San Francisco,USA,2011:313-318.
[15] CAI D,CHEN X L.Large scale spectral clustering via landmark-based sparse representation[J].IEEE Transactions on Cybernetics,2015,45(8):1669-1680.
[16] YE M,LIU W F.Large Scale Spectral Clustering Based on Fast Landmark Sampling[J].Journal of Electronics & Information Technology,2017,39(2):278-284.
[17] ZHANG X C,ZONG L L,YOU Q Z,et al.Sampling forNyström extension based spectral clustering:incremental perspective and novel analysis[J].ACM Transactions on Know-ledge Discovery from Data,2016,11(1):1-25.
[18] CHEN X J,NIE F P,HUANG Z X,et al.Scalable normalized cut with improved spectral rotation[C]//Twenty-Sixth International Joint Conference on Artificial Intelligence.2017:1518-1524.
[19] CHU D R,ZHOU Z P.An autoencoder spectral clustering algorithm for improving landmark representation by weighted PageRank[J].CAAI Transactions on Intelligent Systems,2020,15(2):302-309.
[20] ZHANG M,ZHOU Z P.An autoencoder-based spectral clustering algorithm combined with metric fusion and landmark representation[J].CAAI Transactions on Intelligent Systems,2020,15(4):687-696.
[21] YU S X,SHI J B.Multiclass spectral clustering[C]//Proceedings of the 9th IEEE International Conference on Computer Vision(ICCV'03).USA,2003:313-319.
[22] HUANG J,NIE F P,HUANG H.Spectral rotation versus k-means in spectral clustering[C]//Proceedings of the AAAI Confe rence on Artificial Intelligence(AAAI'13).AAAI Press,2013:431-437.
[23] YANG Y,SHEN F M,HUANG Z,et al.A unified framework for discretespectral clustering[C]//International Joint Confe-rence on Artificial Intelligence.2016:2273-2279.
[24] PANG Y W,XIE J,NIE F P,et al.Spectral clustering by joint spectral embedding and spectral rotation[J].IEEE Transactions on Cybernetics,2020,50(1):247-258.
[25] ZHU J T,JANG J L,LIU T,et al.Joint spectral clustering based on optimal graph and feature selection[J].Neural Processing Letters,2020(11):1-17,
[26] ZHOU L,PING X J,XU S,et al.Cluster Ensemble Based on Spectral Clustering[J].Acta Automatica Sinica,2012,38(8):1335-1342.
[27] STOER M,WAGNER F.A simple min-cut algorithm[J].Journal of the ACM,1997,44(4):585-591.
[28] SHI J B,JITENDRA.Normalized cuts and image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,11(9):888-905.
[29] NG A Y,JORDAN M I,WEISS Y.On spectral clustering:Analysis and an algorithm[C]//Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic.Cambridge,MA,USA,2002:849-856.
[30] LIU W,HE J F,CHANG S F.Large graph construction forscalable semi-supervised learning[C]//Proceeding,Twenty-Se-venth International Conference on Machine Learning.Haifa,Israel,2010:679-686.
[31] JOURNÉE M,NESTEROV Y,RICHTÁRIK P,et al.Genera-lized power method for sparse principal component analysis[J].Journal of Machine Learning Research,2010,11(2):517-553.
[32] DHEERU D,TANISKIDOU E K.UCI Machine Learning Repository[EB/OL].http://archive.ics.uci.edu/ml.
[1] GUO Yi-shan, LIU Man-dan. Anomaly Detection Based on Spatial-temporal Trajectory Data [J]. Computer Science, 2021, 48(6A): 213-219.
[2] ZHANG Xiao-qin, AN Xiao-dan, CAO Fu-yuan. Detecting Community from Bipartite Network Based on Spectral Clustering [J]. Computer Science, 2019, 46(4): 216-221.
[3] LI Chang-jing,ZHAO Shu-liang,CHI Yun-xian. Outlier Detection Algorithm Based on Spectral Embedding and Local Density [J]. Computer Science, 2019, 46(3): 260-266.
[4] LIU Shu-dong, WEI Jia-min. Multilayer Perceptron Classification Algorithm Based on Spectral Clusteringand Simultaneous Two Sample Representation [J]. Computer Science, 2019, 46(11A): 194-198.
[5] WANG Ying and YANG Yu-wang. KNN Similarity Graph Algorithm Based on Heap and Neighborhood Coexistence [J]. Computer Science, 2018, 45(5): 196-200.
[6] CHANG Jia-wei, DAI Mu-hong. Personalized Recommendation Algorithm Based on PageRank and Spectral Method [J]. Computer Science, 2018, 45(11A): 398-401.
[7] LI Peng-qing, LI Yang-ding, DENG Xue-lian, LI Yong-gang, FANG Yue. Spectral Clustering Algorithm Based on SimRank Score [J]. Computer Science, 2018, 45(11A): 458-461.
[8] CHEN Jun-fen, ZHANG Ming, HE Qiang. Heuristically Determining Cluster Numbers Based NJW Spectral Clustering Algorithm [J]. Computer Science, 2018, 45(11A): 474-479.
[9] WANG Ping-xin, LIU Qiang, YANG Xi-bei and MI Ju-sheng. Three-way Clustering Analysis Based on Dynamic Neighborhood [J]. Computer Science, 2018, 45(1): 62-66.
[10] LI Jin-ze, XU Xi-rong, PAN Zi-qi and LI Xiao-jie. Improved Adaptive Spectral Clustering NJW Algorithm [J]. Computer Science, 2017, 44(Z6): 424-427.
[11] LI Xiao-lun and DING Zhi-jun. Group Travel Trip Recommendation Method in LBSNs [J]. Computer Science, 2017, 44(6): 199-205.
[12] QIN Xiao, LIANG Wei, YUAN Chang-an and TANG Tao. Image Segmentation Algorithm of Spectral Clustering Optimized by Genetic [J]. Computer Science, 2017, 44(1): 100-102.
[13] FAN Jing, RUAN Ti-hong, WU Jia-min and DONG Tian-yang. Vehicle Behavior Recognition Method Based on Quadratic Spectral Clustering and HMM-RF Hybrid Model [J]. Computer Science, 2016, 43(5): 288-293.
[14] WANG You-hua and CHEN Xiao-rong. Improved Text Clustering Algorithm Based on Kolmogorov Complexity [J]. Computer Science, 2016, 43(5): 243-246.
[15] ZHOU Hai-song and HUANG De-cai. Density Self-adaption Semi-supervised Spectral Clustering Algorithm [J]. Computer Science, 2016, 43(12): 209-212.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!