Computer Science ›› 2021, Vol. 48 ›› Issue (6): 79-85.doi: 10.11896/jsjkx.200900014

• Database & Big Data & Data Science • Previous Articles     Next Articles

Kernel-preserving Embedding Based Subspace Learning

HE Wen-qi1,2,3, LIU Bao-long1,3, SUN Zhao-chuan3, WANG Lei1,2,3, LI Dan-ping4   

  1. 1 Xidian University QingdaoInstitute of Computing Technology,Qingdao,Shandong 266000,China
    2 Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education,Shanghai Jiao Tong University,Shanghai 200240,China
    3 School of Electronic Engineering,Xidian University,Xi’an 710071,China
    4 School of Telecommunications Engineering,Xidian University,Xi’an 710071,China
  • Received:2020-09-02 Revised:2020-11-07 Online:2021-06-15 Published:2021-06-03
  • About author:HE Wen-qi,born in 1996,postgraduate,is a member of China Computer Federation.Her main research interests include subspace learning and multi-view learning.(18852143390@163.com)
    LI Dan-ping,born in 1981,Ph.D,lectu-rer.Her main research interests include wireless signal processing and machine learning.
  • Supported by:
    National Key Research and Development Program of China(2016YFE0207000),National Natural Science Foundation of China(61203137,61401328) and Natural Science Basic Research Program of Shanxi Province of China(2014JQ8306,2015JM6279).

Abstract: Subspace learning is an important research subject in the field of feature extraction.It maps the original data into a low-dimensional subspace through a linear or nonlinear transformation,and preserves the geometric structure and useful information of the original data as much as possible in this subspace.The performance of subspace learning mainly depends on the design of similarity measure and the graph construction for feature embedding.Aiming at the two issues,a novel kernel-preserving embedding based subspace learning(KESL) method is proposed,which can adaptively learn the similarity information from data and construct the kernel-preserving graph.First,to tackle the problem that the traditional dimension reduction methods cannot preserve the inner structure of high-dimensional nonlinear data,our algorithm introduces the kernel function and minimizes the reconstruction error of samples,which is beneficial for mining the data structural relationship for classification.Then,aiming at the limitation that existing graph-based subspace learning methods mainly concern the similarity information of the samples within a class,our algorithm uses the learned similarity matrices to construct intra-class and inter-class graphs,respectively.Thus,in the projected subspace,the kernel-preserving relationship of the samples in the same class can be strengthened,while the kernel-preserving relationship of the samples from different classes can be largely inhibited.Finally,through the joint optimization of kernel preserving matrix and graph embedding,the desired projection under the optimal representation can be dynamically solved.Expe-rimental results on several datasets show that the proposed algorithm is competitive to the state-of-the-art subspace learning algorithms in various classification tasks.

Key words: Graph construction, Kernel-preserving embedding, Similarity learning, Subspace learning

CLC Number: 

  • TP391
[1]CHEN X J,YE Y M,XU X F,et al.A feature group weighting method for subspace clustering of high-dimensional data[J].Pattern Recognition,2012,45(1):434-446.
[2]HUANG S,ELGAMMAL A M,YANG D.Learning Speed Invariant Gait Template via Thin Plate Spline Kernel Manifold Fitting[C]//British Machine Vision Conference 2013.2013.
[3]LAI Z H,BAO J Q,KONG H,et al.Discriminative low-rank projection for robust subspace learning[J].International Journal of Machine Learning and Cybernetics,2020,11(5):2247-2260.
[4]LI J X,ZHAO Z G,LI Q,et al.Improved Locality and Similarity Preserving Feature Selection Algorithm[J].Computer Science,2020,47(S1):480-484.
[5]WOLD S,ESBENSEN K H,GELADI P,et al.Principal Component Analysis[J].Chemometrics and Intelligent Laboratory Systems,1987,2(1/2/3):37-52.
[6]FISHER R A.The use of multiple measurements in taxonomic problems[J].Annals of Human Genetics,1936,7(2):179-188.
[7]KIM K I,JUNG K,KIM H J,et al.Face recognition using kernel principal component analysis[J].IEEE Signal Processing Letters,2002,9(2):40-42.
[8]MIKA S,RATSCH G,WESTON J,et al.Fisher discriminantanalysis with kernels[C]//Neural Networks for Signal Proces-sing Ix,1999.Proceedings of the IEEE Signal Processing Society Workshop.IEEE,1999:41-48.
[9]YAN S C,XU D,ZHANG B Y,et al.Graph Embedding and Extensions:A General Framework for Dimensionality Reduction[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(1):40-51.
[10]XUE X Y,MA X H.Double Adjacency Graphs Based Orthogonal Neighborhood Preserving Projections for Face Recognition[J].Computer Science,2017,44(8):31-35.
[11]YIN M,GAO J,LIN Z,et al.Laplacian Regularized Low-Rank Representation and Its Applications[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(3):504-517.
[12]NG A Y,JORDAN M I,WEISS Y,et al.On Spectral Cluste-ring:Analysis and an algorithm[C]//Neural Information Processing Systems.2001:849-856.
[13]WEINBERGER K Q,SAUL L K.Distance Metric Learning for Large Margin Nearest Neighbor Classification[J].Journal of Machine Learning Research,2009,10(1):207-244.
[14]JIAO X,CHEN Y G,DONG R.An Unsupervised Image Segmentation Method Combining Graph Clustering and High-Level Feature Representation[J].Neurocomputing,2020,409(7):83-92.
[15]HIRZER M,ROTH P M,KÖSTINGER M,et al.Relaxed pairwise learned metric for person re-identification[C]//European Conference on Computer Vision.Berlin,Heidelberg:Springer,2012:780-793.
[16]HOI S C,LIU W,CHANG S,et al.Semi-supervised distancemetric learning for Collaborative Image Retrieval[C]//Compu-ter Vision and Pattern Recognition.2008:1-7.
[17]ZHANG L,YANG M,FENG X,et al.Collaborative Representation based Classification for Face Recognition[J].arXiv:1204.2358,2012.
[18]TAO Z,LIU H,LI S,et al.From Ensemble Clustering to Multi-View Clustering[C]//International Joint Conference on Artificial Intelligence.2017:2843-2849.
[19]WANG L,LI M,JI H,et al.When collaborative representation meets subspace projection:A novel supervised framework of graph construction augmented by anti-collaborative representation[J].Neurocomputing,2019,328:157-170.
[20]WRIGHT J,YANG A Y,GANESH A,et al.Robust Face Recog-nition via Sparse Representation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(2):210-227.
[21]QIAO L S,CHEN S C,TAN X Y,et al.Sparsity preserving projections with applications to face recognition[J].Pattern Reco-gnition,2010,43(1):331-341.
[22]ZHANG L,YANG M,FENG X C,et al.Sparse representation or collaborative representation:Which helps face recognition? [C]//International Conference on Computervision.2011:471-478.
[23]LY N H,DU Q,FOWLER J E.Collaborative graph-based discriminant analysis for hyperspectral imagery[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2014,7(6):2688-2696.
[24]PENG X,YU Z,YI Z,et al.Constructing the L2-Graph for Robust Subspace Learning and Subspace Clustering[J].IEEE Transactions on Systems,Man,and Cybernetics,2017,47(4):1053-1066.
[25]PENG X,LU J,YI Z,et al.Automatic Subspace Learning via Principal Coefficients Embedding[J].IEEE Transactions on Systems,Man,and Cybernetics,2017,47(11):3583-3596.
[26]WEN J,HAN N,FANG X,et al.Low-Rank Preserving Projec-tion Via Graph Regularized Reconstruction[J].IEEE Transac-
tions on Systems,Man,and Cybernetics,2019,49(4):1279-1291.
[27]KANG Z,LU Y,SU Y,et al.Similarity Learning via Kernel Preserving Embedding[C]//National Conference on Artificial Intelligence.2019:4057-4064.
[28]WEN J,ZHANG B,XU Y,et al.Adaptive weighted nonnegative low-rank representation[J].Pattern Recognition,2018,81:326-340.
[29]SAMARIA F S,HARTER A.Parameterisation of a stochastic model for human face identification[C]//Workshop on Applications of Computer Vision.1994:138-142.
[30]GRAHAM D B,ALLINSON N M.Characterising virtual eigensignatures for general purpose face recognition[M].Berlin,Heidelberg:Springer,1998:446-456.
[1] DENG Kai, YANG Pin, LI Yi-zhou, YANG Xing, ZENG Fan-rui, ZHANG Zhen-yu. Fast and Transmissible Domain Knowledge Graph Construction Method [J]. Computer Science, 2022, 49(6A): 100-108.
[2] CAO He-xin, ZHAO Liang, LI Xue-feng. Technical Research of Graph Neural Network for Text-to-SQL Parsing [J]. Computer Science, 2022, 49(4): 110-115.
[3] LIANG Jing-ru, E Hai-hong, Song Mei-na. Method of Domain Knowledge Graph Construction Based on Property Graph Model [J]. Computer Science, 2022, 49(2): 174-181.
[4] YANG Lei, JIANG Ai-lian, QIANG Yan. Structure Preserving Unsupervised Feature Selection Based on Autoencoder and Manifold Regularization [J]. Computer Science, 2021, 48(8): 53-59.
[5] BAI Zi-yi, MAO Yi-rong , WANG Rui-ping. Survey on Video-based Face Recognition [J]. Computer Science, 2021, 48(3): 50-59.
[6] ZHANG Han-shuo, YANG Dong-ju. Technology Data Analysis Algorithm Based on Relational Graph [J]. Computer Science, 2021, 48(3): 174-179.
[7] XU Shu-yan, HAN Li-xin, XU Guo-xia. Domain Adaptation Algorithm Based on Tensor Decomposition [J]. Computer Science, 2019, 46(12): 89-94.
[8] YUAN Min,YANG Rui-guo,YUAN Yuan and LEI Ying-ke. New Supervised Manifold Learning Method Based on MMC [J]. Computer Science, 2014, 41(4): 273-279.
[9] DU Hai-shun,LI Yu-ling,WANG Feng-quan,ZHANG Fan. Face Recognition Using Kernel Maximum Scatter Difference Discriminant Analysis [J]. Computer Science, 2010, 37(6): 286-288.
[10] YU Cheng-Wen, GUO Lei,  ZHANG Qian-Jin, LI Hui-Hui (College of Automation, Northwestern Polytechnical University, Xi'an 710072). [J]. Computer Science, 2007, 34(8): 219-222.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!