计算机科学 ›› 2021, Vol. 48 ›› Issue (6): 79-85.doi: 10.11896/jsjkx.200900014

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于核保持嵌入的子空间学习

贺文琪1,2,3, 刘保龙1,3, 孙兆川3, 王磊1,2,3, 李丹萍4   

  1. 1 西安电子科技大学青岛计算技术研究院 山东 青岛266000
    2 上海交通大学海洋智能装备与系统教育部重点实验室 上海200240
    3 西安电子科技大学电子工程学院 西安710071
    4 西安电子科技大学通信工程学院 西安710071
  • 收稿日期:2020-09-02 修回日期:2020-11-07 出版日期:2021-06-15 发布日期:2021-06-03
  • 通讯作者: 李丹萍(dpli@xidian.edu.cn)
  • 基金资助:
    国家重点研发计划(2016YFE0207000);国家自然科学基金(61203137,61401328);陕西省自然科学基础研究计划资助项目(2014JQ8306,2015JM6279)

Kernel-preserving Embedding Based Subspace Learning

HE Wen-qi1,2,3, LIU Bao-long1,3, SUN Zhao-chuan3, WANG Lei1,2,3, LI Dan-ping4   

  1. 1 Xidian University QingdaoInstitute of Computing Technology,Qingdao,Shandong 266000,China
    2 Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education,Shanghai Jiao Tong University,Shanghai 200240,China
    3 School of Electronic Engineering,Xidian University,Xi’an 710071,China
    4 School of Telecommunications Engineering,Xidian University,Xi’an 710071,China
  • Received:2020-09-02 Revised:2020-11-07 Online:2021-06-15 Published:2021-06-03
  • About author:HE Wen-qi,born in 1996,postgraduate,is a member of China Computer Federation.Her main research interests include subspace learning and multi-view learning.(18852143390@163.com)
    LI Dan-ping,born in 1981,Ph.D,lectu-rer.Her main research interests include wireless signal processing and machine learning.
  • Supported by:
    National Key Research and Development Program of China(2016YFE0207000),National Natural Science Foundation of China(61203137,61401328) and Natural Science Basic Research Program of Shanxi Province of China(2014JQ8306,2015JM6279).

摘要: 子空间学习是特征提取领域中的一个重要研究方向,其通过一种线性或非线性的变换将原始数据映射到低维子空间中,并在该子空间中尽可能地保留原始数据的几何结构和有用信息。子空间学习的性能提升主要取决于相似性关系的衡量方式和特征嵌入的图构建手段。文中针对子空间学习中的相似性度量与图构建两大问题进行研究,提出了一种基于核保持嵌入的子空间学习算法(Kernel-preserving Embedding based Subspace Learning,KESL),该算法通过自表示技术自适应地学习数据间的相似性信息和基于核保持的构图。首先针对传统降维方法无法挖掘高维非线性数据的内部结构问题,引入核函数并最小化样本的重构误差来约束最优的表示系数,以期挖掘出有利于分类的数据结构关系。然后,针对现有基于图的子空间学习方法大都只考虑类内样本相似性信息的问题,利用学习到的相似性矩阵分别构建类内和类间图,使得在投影子空间中同类样本的核保持关系得到加强,不同类样本间的核保持关系被进一步抑制。最后,通过核保持矩阵与图嵌入的联合优化,动态地求解出最优表示下的子空间投影。在多个数据集上的实验结果表明,所提算法在分类任务中的性能优于主流的子空间学习算法。

关键词: 核保持嵌入, 图构建, 相似性学习, 子空间学习

Abstract: Subspace learning is an important research subject in the field of feature extraction.It maps the original data into a low-dimensional subspace through a linear or nonlinear transformation,and preserves the geometric structure and useful information of the original data as much as possible in this subspace.The performance of subspace learning mainly depends on the design of similarity measure and the graph construction for feature embedding.Aiming at the two issues,a novel kernel-preserving embedding based subspace learning(KESL) method is proposed,which can adaptively learn the similarity information from data and construct the kernel-preserving graph.First,to tackle the problem that the traditional dimension reduction methods cannot preserve the inner structure of high-dimensional nonlinear data,our algorithm introduces the kernel function and minimizes the reconstruction error of samples,which is beneficial for mining the data structural relationship for classification.Then,aiming at the limitation that existing graph-based subspace learning methods mainly concern the similarity information of the samples within a class,our algorithm uses the learned similarity matrices to construct intra-class and inter-class graphs,respectively.Thus,in the projected subspace,the kernel-preserving relationship of the samples in the same class can be strengthened,while the kernel-preserving relationship of the samples from different classes can be largely inhibited.Finally,through the joint optimization of kernel preserving matrix and graph embedding,the desired projection under the optimal representation can be dynamically solved.Expe-rimental results on several datasets show that the proposed algorithm is competitive to the state-of-the-art subspace learning algorithms in various classification tasks.

Key words: Graph construction, Kernel-preserving embedding, Similarity learning, Subspace learning

中图分类号: 

  • TP391
[1]CHEN X J,YE Y M,XU X F,et al.A feature group weighting method for subspace clustering of high-dimensional data[J].Pattern Recognition,2012,45(1):434-446.
[2]HUANG S,ELGAMMAL A M,YANG D.Learning Speed Invariant Gait Template via Thin Plate Spline Kernel Manifold Fitting[C]//British Machine Vision Conference 2013.2013.
[3]LAI Z H,BAO J Q,KONG H,et al.Discriminative low-rank projection for robust subspace learning[J].International Journal of Machine Learning and Cybernetics,2020,11(5):2247-2260.
[4]LI J X,ZHAO Z G,LI Q,et al.Improved Locality and Similarity Preserving Feature Selection Algorithm[J].Computer Science,2020,47(S1):480-484.
[5]WOLD S,ESBENSEN K H,GELADI P,et al.Principal Component Analysis[J].Chemometrics and Intelligent Laboratory Systems,1987,2(1/2/3):37-52.
[6]FISHER R A.The use of multiple measurements in taxonomic problems[J].Annals of Human Genetics,1936,7(2):179-188.
[7]KIM K I,JUNG K,KIM H J,et al.Face recognition using kernel principal component analysis[J].IEEE Signal Processing Letters,2002,9(2):40-42.
[8]MIKA S,RATSCH G,WESTON J,et al.Fisher discriminantanalysis with kernels[C]//Neural Networks for Signal Proces-sing Ix,1999.Proceedings of the IEEE Signal Processing Society Workshop.IEEE,1999:41-48.
[9]YAN S C,XU D,ZHANG B Y,et al.Graph Embedding and Extensions:A General Framework for Dimensionality Reduction[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(1):40-51.
[10]XUE X Y,MA X H.Double Adjacency Graphs Based Orthogonal Neighborhood Preserving Projections for Face Recognition[J].Computer Science,2017,44(8):31-35.
[11]YIN M,GAO J,LIN Z,et al.Laplacian Regularized Low-Rank Representation and Its Applications[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(3):504-517.
[12]NG A Y,JORDAN M I,WEISS Y,et al.On Spectral Cluste-ring:Analysis and an algorithm[C]//Neural Information Processing Systems.2001:849-856.
[13]WEINBERGER K Q,SAUL L K.Distance Metric Learning for Large Margin Nearest Neighbor Classification[J].Journal of Machine Learning Research,2009,10(1):207-244.
[14]JIAO X,CHEN Y G,DONG R.An Unsupervised Image Segmentation Method Combining Graph Clustering and High-Level Feature Representation[J].Neurocomputing,2020,409(7):83-92.
[15]HIRZER M,ROTH P M,KÖSTINGER M,et al.Relaxed pairwise learned metric for person re-identification[C]//European Conference on Computer Vision.Berlin,Heidelberg:Springer,2012:780-793.
[16]HOI S C,LIU W,CHANG S,et al.Semi-supervised distancemetric learning for Collaborative Image Retrieval[C]//Compu-ter Vision and Pattern Recognition.2008:1-7.
[17]ZHANG L,YANG M,FENG X,et al.Collaborative Representation based Classification for Face Recognition[J].arXiv:1204.2358,2012.
[18]TAO Z,LIU H,LI S,et al.From Ensemble Clustering to Multi-View Clustering[C]//International Joint Conference on Artificial Intelligence.2017:2843-2849.
[19]WANG L,LI M,JI H,et al.When collaborative representation meets subspace projection:A novel supervised framework of graph construction augmented by anti-collaborative representation[J].Neurocomputing,2019,328:157-170.
[20]WRIGHT J,YANG A Y,GANESH A,et al.Robust Face Recog-nition via Sparse Representation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(2):210-227.
[21]QIAO L S,CHEN S C,TAN X Y,et al.Sparsity preserving projections with applications to face recognition[J].Pattern Reco-gnition,2010,43(1):331-341.
[22]ZHANG L,YANG M,FENG X C,et al.Sparse representation or collaborative representation:Which helps face recognition? [C]//International Conference on Computervision.2011:471-478.
[23]LY N H,DU Q,FOWLER J E.Collaborative graph-based discriminant analysis for hyperspectral imagery[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2014,7(6):2688-2696.
[24]PENG X,YU Z,YI Z,et al.Constructing the L2-Graph for Robust Subspace Learning and Subspace Clustering[J].IEEE Transactions on Systems,Man,and Cybernetics,2017,47(4):1053-1066.
[25]PENG X,LU J,YI Z,et al.Automatic Subspace Learning via Principal Coefficients Embedding[J].IEEE Transactions on Systems,Man,and Cybernetics,2017,47(11):3583-3596.
[26]WEN J,HAN N,FANG X,et al.Low-Rank Preserving Projec-tion Via Graph Regularized Reconstruction[J].IEEE Transac-
tions on Systems,Man,and Cybernetics,2019,49(4):1279-1291.
[27]KANG Z,LU Y,SU Y,et al.Similarity Learning via Kernel Preserving Embedding[C]//National Conference on Artificial Intelligence.2019:4057-4064.
[28]WEN J,ZHANG B,XU Y,et al.Adaptive weighted nonnegative low-rank representation[J].Pattern Recognition,2018,81:326-340.
[29]SAMARIA F S,HARTER A.Parameterisation of a stochastic model for human face identification[C]//Workshop on Applications of Computer Vision.1994:138-142.
[30]GRAHAM D B,ALLINSON N M.Characterising virtual eigensignatures for general purpose face recognition[M].Berlin,Heidelberg:Springer,1998:446-456.
[1] 曹合心, 赵亮, 李雪峰.
图神经网络在Text-to-SQL解析中的技术研究
Technical Research of Graph Neural Network for Text-to-SQL Parsing
计算机科学, 2022, 49(4): 110-115. https://doi.org/10.11896/jsjkx.210200173
[2] 杨蕾, 降爱莲, 强彦.
基于自编码器和流形正则的结构保持无监督特征选择
Structure Preserving Unsupervised Feature Selection Based on Autoencoder and Manifold Regularization
计算机科学, 2021, 48(8): 53-59. https://doi.org/10.11896/jsjkx.200700211
[3] 白子轶, 毛懿荣, 王瑞平.
视频人脸识别进展综述
Survey on Video-based Face Recognition
计算机科学, 2021, 48(3): 50-59. https://doi.org/10.11896/jsjkx.210100210
[4] 齐少华, 徐和根, 万友文, 付豪.
动态环境下的语义地图构建
Construction of Semantic Mapping in Dynamic Environments
计算机科学, 2020, 47(9): 198-203. https://doi.org/10.11896/jsjkx.191000040
[5] 王秉洲, 王慧斌, 沈洁, 张丽丽.
基于自适应渐消UKF的FastSLAM算法
FastSLAM Algorithm Based on Adaptive Fading Unscented Kalman Filter
计算机科学, 2020, 47(9): 213-218. https://doi.org/10.11896/jsjkx.190700186
[6] 徐书艳, 韩立新, 徐国夏.
基于张量分解的域适应算法
Domain Adaptation Algorithm Based on Tensor Decomposition
计算机科学, 2019, 46(12): 89-94. https://doi.org/10.11896/jsjkx.190300095
[7] 罗元,苏琴,张毅,管国伦.
基于迭代无迹H∞滤波的移动机器人SLAM
Iterated Unscented H∞ Filter Based Mobile Robot SLAM
计算机科学, 2017, 44(8): 306-311. https://doi.org/10.11896/j.issn.1002-137X.2017.08.053
[8] 袁暋,杨瑞国,原媛,雷迎科.
一种新的基于最大边缘准则的监督流形学习方法
New Supervised Manifold Learning Method Based on MMC
计算机科学, 2014, 41(4): 273-279.
[9] 杜海顺,李玉玲,汪凤泉,张帆.
一种核最大散度差判别分析人脸识别方法
Face Recognition Using Kernel Maximum Scatter Difference Discriminant Analysis
计算机科学, 2010, 37(6): 286-288.
[10] 杨锦园,黄心汉,李鹏.
基于DSmT的移动机器人地图构建及传感器管理
DSmT-based Mobile Robot Map Building and Sensor Management
计算机科学, 2010, 37(4): 227-.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!