计算机科学 ›› 2022, Vol. 49 ›› Issue (2): 191-197.doi: 10.11896/jsjkx.210300034

• 数据库&大数据&数据科学 • 上一篇    下一篇

鲁棒联合稀疏不相关回归

李宗然1, 陈秀宏1,2, 陆赟1, 邵政毅1   

  1. 1 江南大学人工智能与计算机学院 江苏 无锡214122
    2 江苏省媒体设计与软件技术重点实验室 江苏 无锡214122
  • 收稿日期:2021-03-02 修回日期:2021-07-10 出版日期:2022-02-15 发布日期:2022-02-23
  • 通讯作者: 陈秀宏(xiuhongc@jiangnan.edu.cn)
  • 作者简介:18851656843@163.com

Robust Joint Sparse Uncorrelated Regression

LI Zong-ran1, CHEN XIU-Hong1,2, LU Yun1, SHAO Zheng-yi1   

  1. 1 School of Artificial Intelligence and Computer Science,Jiangnan University,Wuxi,Jiangsu 214122,China
    2 Jiangsu Key Laboratory of Media Design and Software Technology,Wuxi,Jiangsu 214122,China
  • Received:2021-03-02 Revised:2021-07-10 Online:2022-02-15 Published:2022-02-23
  • About author:LI Zong-Ran,born in 1995,postgra-duate.His main research interests include machine learning,pattern recognition and digital image processing.
    CHEN Xiu-hong,born in 1964,Ph.D.His main research interests include di-gital image processing,pattern recognition,target detection and tracking,optimization theory and method and so on.

摘要: 常见的无监督特征选择方法考虑的只是选择具有判别性的特征,而忽略了特征的冗余性,并且没有考虑到小类问题,故而影响到分类性能。基于此背景,提出鲁棒不相关回归算法。首先,对不相关回归进行研究,使用不相关正交约束,以便找出不相关但具有判别性的特征,不相关约束使得数据结构保持在Stiefel流形中,使模型具有封闭解,避免了传统的岭回归模型引发的可能的平凡解。其次,损失函数与正则化项使用L2,1范数,保证模型的鲁棒性,得到具有稀疏性的投影矩阵;同时将小类问题考虑进去,使投影矩阵数量不受类别数的限制,得到足够多的投影矩阵,从而提升模型的分类性能。理论分析和多个数据集上的实验结果表明,所提出的方法比其他特征选择方法具有更好的性能。

关键词: 不相关, 回归, 联合, 鲁棒, 特征选择, 小类问题

Abstract: Common unsupervised feature selection methods only consider the selection of discriminative features,while ignoring the redundancy of features and failing to consider the problem of small classes,which affect the classification performance.Based on this background,a robust uncorrelated regression algorithm is proposed.First,research on uncorrelated regression,use uncorrelated orthogonal constraints to find irrelevant but discriminative features.Uncorrelated constraints keep the data structure in the Stiefel manifold,making the model have a closed solution,avoiding the possible trivial solutions caused by the traditional ridge regression model.Secondly,the loss function and the regularization term use the L2,1 norm to ensure the robustness of the model and obtain a sparse projection matrix.At the same time,the small class problem is taken into account,so that the number of projection matrices is not limited by the number of classes,and the result is enough projection matrices to improve the classification performance of the model.Theoretical analysis and experimental results on multiple data sets show that the proposed method has better performance than other feature selection methods.

Key words: Feature selection, Joint, Regression, Robust, Small-class, Uncorrelated

中图分类号: 

  • TP391.4
[1]MAATEN L,POSTMA E,HERIK J.Dimensionality reduction:A comparative review[J].Review Literature & Arts of the Americas,2009,10(1):158-165.
[2]JAIN A,ZONGKER D.Feature selection:evaluation,applica-tion,and small sample performance[J].IEEE Trans. Pattern Anal. Mach. Intell,1997,19(2):153-158.
[3]DUDA R O,HART P E,STORK D G.Pattern Classification[M].Wiley,2001:63-69.
[4]BELHUMEUR P N,HESPANHA J P,KRIEGMAN D J.Eigenfaces vs.Fisherfaces:Recognition Using Class Specific Li-near Projection[C]//European Conference on Computer Vision.Springer-Verlag,1996:15-28.
[5]HE J,DING L,LEI J,et al.Kernel ridge regression classification[C]//Proceedings of International Joint Conference on Neural Networks.IEEE,2014:137-146.
[6]NIE F,HUANG H,XIAO C,et al.Efficient and Robust Feature Selection via Joint ℓ2,1-Norms Minimization[C]//International Conference on Neural Information Processing Systems.Curran Associates Inc.,2010:1813-1821.
[7]CHEN X H,LU Y.Robust Graph Regularized Sparse Matrix Regression for Two-dimensional Supervised Feature Selection[J].IET Image Processing,2020,14(9):1740-1749.
[8]MO D,LAI Z.Robust Jointly Sparse Regression with Genera-lized Orthogonal Learning for Image Feature Selection[J].Pattern Recognition,2019,16(3):964-985.
[9]HOU C,JIAO Y,NIE F,et al.2D Feature Selection by SparseMatrix Regression[J].IEEE Transactions on Image Processing,2017,26(9):4255-4268.
[10]CHEN X H,LU Y.Dynamic Graph Regularization and Label Relaxation- Based Sparse Matrix Regression for Two-dimensional Feature Selection[J].IEEE Access,2020,8:62855-62870.
[11]YU Y,CHEN X.Joint feature weighting and adaptive graph-based matrix regression for image supervised feature Selection-ScienceDirect[J].Signal Processing:Image Communication,2020,13(2):9-16.
[12]YANG L,JIANG A L,QIANG Y.Structure Preserving Unsupervised Feature Selection Based on Autoencoder and Manifold Regularization[J].Computer Science,2021,48(8):53-59.
[13]YI Y,SHEN H T,MA Z,et al.L21-norm regularized discriminative feature selection for unsupervised learning[C]//Procee-dings of the Twenty-Second international joint conference on Artificial Intelligence.AAAI Press,2011:56-59.
[14]HOU C,NIE F,LI X,et al.Joint Embedding Learning andSparse Regression:A Framework for Unsupervised Feature Selection[J].IEEE Transactions on Cybernetics,2013,44(6):793-804.
[15]LIU X,WANG L,ZHANG J,et al.Global and Local Structure Preservation for Feature Selection[J].IEEE Transactions on Neural Networks and Learning Systems,2017,25(6):1083-1095.
[16]YUAN H,LI J,LAI L L,et al.Joint sparse matrix regression and nonnegative spectral analysis for two-dimensional unsupervised feature selection[J].Pattern Recognition,2019,66(3):119-133.
[17]LI X,ZHANG H,ZHANG R,et al.Generalized UncorrelatedRegression with Adaptive Graph for Unsupervised Feature Selection[J].IEEE Transactions on Neural Networks and Lear-ning Systems,2018,3(6):1-9.
[18]ZOU H,TIBSHIRANI H R.Sparse Principal Component Ana-lysis[J].Journal of Computational and Graphical Stats,2006,15(2):1-30.
[19]DAVENPORT M A,WAKIN M B.Analysis of OrthogonalMatching Pursuit using the Restricted Isometry Property[J].IEEE Transactions on Information Theory,2010,56(9):4395-4401.
[20]NIE F,ZHANG R,LI X.A generalized power iteration method for solving quadratic problem on the Stiefel manifold[J].Science China,2017(11):146-155.
[21]BELHUMEUR P N,HESPANHA J P,KRIEGMAN D J.Eigenfaces vs.Fisherfaces:Recognition Using Class Specific Li-near Projection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1997,19(7):711-720.
[22]GEORGHIADES A,BELHUMEUR P P,KRIEGMAN D.Yale face database[OL].http://cvc.yale.edu/ptojects/yalefaces/yalefaces.html.
[23]JESORSKY O.Robust Face Detection Using the Hausdorff Distance[J].AVBPA,2001,32(10):90-95.
[24]LYONS M J,BUDYNEK J.Automatic classification of single facial images[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1999,4(3):93-104.
[25]NENE S A.Columbia Object Image Library(COIL-20)[J].Technical Report,1996,5(8):63-68.
[1] 吕由, 吴文渊.
隐私保护线性回归方案与应用
Privacy-preserving Linear Regression Scheme and Its Application
计算机科学, 2022, 49(9): 318-325. https://doi.org/10.11896/jsjkx.220300190
[2] 李斌, 万源.
基于相似度矩阵学习和矩阵校正的无监督多视角特征选择
Unsupervised Multi-view Feature Selection Based on Similarity Matrix Learning and Matrix Alignment
计算机科学, 2022, 49(8): 86-96. https://doi.org/10.11896/jsjkx.210700124
[3] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[4] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[5] 康雁, 王海宁, 陶柳, 杨海潇, 杨学昆, 王飞, 李浩.
混合改进的花授粉算法与灰狼算法用于特征选择
Hybrid Improved Flower Pollination Algorithm and Gray Wolf Algorithm for Feature Selection
计算机科学, 2022, 49(6A): 125-132. https://doi.org/10.11896/jsjkx.210600135
[6] 陈永平, 朱建清, 谢懿, 吴含笑, 曾焕强.
基于外接圆半径差损失的实时安全帽检测算法
Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss
计算机科学, 2022, 49(6A): 424-428. https://doi.org/10.11896/jsjkx.220100252
[7] 闫萌, 林英, 聂志深, 曹一凡, 皮欢, 张兰.
一种提高联邦学习模型鲁棒性的训练方法
Training Method to Improve Robustness of Federated Learning
计算机科学, 2022, 49(6A): 496-501. https://doi.org/10.11896/jsjkx.210400298
[8] 王文强, 贾星星, 李朋.
自适应的集成定序算法
Adaptive Ensemble Ordering Algorithm
计算机科学, 2022, 49(6A): 242-246. https://doi.org/10.11896/jsjkx.210200108
[9] 李京泰, 王晓丹.
基于代价敏感激活函数XGBoost的不平衡数据分类方法
XGBoost for Imbalanced Data Based on Cost-sensitive Activation Function
计算机科学, 2022, 49(5): 135-143. https://doi.org/10.11896/jsjkx.210400064
[10] 赵亮, 张洁, 陈志奎.
基于双图正则化的自适应多模态鲁棒特征学习
Adaptive Multimodal Robust Feature Learning Based on Dual Graph-regularization
计算机科学, 2022, 49(4): 124-133. https://doi.org/10.11896/jsjkx.210300078
[11] 储安琪, 丁志军.
基于灰狼优化算法的信用评估样本均衡化与特征选择同步处理
Application of Gray Wolf Optimization Algorithm on Synchronous Processing of Sample Equalization and Feature Selection in Credit Evaluation
计算机科学, 2022, 49(4): 134-139. https://doi.org/10.11896/jsjkx.210300075
[12] 孙林, 黄苗苗, 徐久成.
基于邻域粗糙集和Relief的弱标记特征选择方法
Weak Label Feature Selection Method Based on Neighborhood Rough Sets and Relief
计算机科学, 2022, 49(4): 152-160. https://doi.org/10.11896/jsjkx.210300094
[13] 赵越, 余志斌, 李永春.
基于互注意力指导的孪生跟踪算法
Cross-attention Guided Siamese Network Object Tracking Algorithm
计算机科学, 2022, 49(3): 163-169. https://doi.org/10.11896/jsjkx.210300066
[14] 张程瑞, 陈俊杰, 郭浩.
静息态人脑功能超网络模型鲁棒性对比分析
Comparative Analysis of Robustness of Resting Human Brain Functional Hypernetwork Model
计算机科学, 2022, 49(2): 241-247. https://doi.org/10.11896/jsjkx.201200067
[15] 肖康, 周夏冰, 王中卿, 段湘煜, 周国栋, 张民.
基于产品建模的评论问题生成研究
Review Question Generation Based on Product Profile
计算机科学, 2022, 49(2): 272-278. https://doi.org/10.11896/jsjkx.201200208
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!