计算机科学 ›› 2018, Vol. 45 ›› Issue (6A): 151-156.

• 模式识别与图像处理 • 上一篇    下一篇

改进型低秩线性回归新方法

于传波,聂仁灿,周冬明,黄帆,丁婷婷   

  1. 云南大学信息学院 昆明650500
  • 出版日期:2018-06-20 发布日期:2018-08-03
  • 作者简介:于传波(1991-),男,硕士生,主要研究方向为模式识别、图像处理,E-mail:18213018302@163.com;聂仁灿(1982-),男,博士,副教授,主要研究方向为图像处理、机器学习等;周冬明(1963-),男,博士,教授,主要研究方向为人工神经网络、优化计算、信号处理技术等,E-mail:zhoudm@ynu.edu.cn(通信作者)。
  • 基金资助:
    国家自然科学基金(61365001,61463052)资助

Novel Method of Improved Low Rank Linear Regression

YU Chuan-bo,NIE Ren-can,ZHOU Dong-ming, HUANG Fan, DING Ting-ting   

  1. School of Information Science and Engineering,Yunnan University,Kunming 650500,China
  • Online:2018-06-20 Published:2018-08-03

摘要: 对于遮挡、光照等影响因素,低秩线性回归模型具有很好的鲁棒性。LRRR(Low Rank Ridge Regression)以及DENLR(Discriminative Elastic-net Regularized Linear Regression)通过正则化系数矩阵在一定程度上减少了LRLR(Low Rank Linear Regression)产生的过拟合现象。但其没有考虑子空间数据的错误逼近,投影矩阵不能准确地将数据映射到目标空间。鉴于此,提出了一种运算更快、更具判别性的低秩线性回归分类新方法。首先,将0-1构成的矩阵作为线性回归的目标值;其次,利用核范数作为低秩约束的凸近似;然后,通过正则化各类别之间的距离矩阵和模型输出矩阵来降低过拟合,同时可以增强投影子空间的判别性;再次,利用增广拉格朗日乘子(Augmented Lagrangian Multiplier,ALM)优化目标函数;最后,在子空间中利用最近邻分类器进行分类。在AR、FERET人脸数据库、Stanford 40 Actions、Caltech-UCSD Bird以及Oxford 102 Flowers数据库上进行相关算法的对比实验,结果表明所提算法是有效的。

关键词: 低秩, 分类, 线性回归, 正则化

Abstract: The low rank linear regression model has good robustness for the influence of occlusion and illumination and so on.To a certain extent,the overfitting phenomenon of LRLR (Low Rank Linear Regression) is reduced in LRRR (Low Rank Ridge Regression) and DENLR (Discriminative Elastic-Net Regularized Linear Regression) by regularization coefficient matrix.Because the error approximation of data in subspace is ill-considered,the data are hardly mapped to the target space accurately via projection matrix.This paper proposed has low rank linear regression classification method which has a faster computing speed and is more discriminative.Firstly,the 0-1 constitutive matrix is regarded as the target value of the linear regression.Secondly,the kernel norm is used as the convex approximation of low rank constraints.Thirdly,all kinds of the distance matrix and the model output matrix are regularized to reduce overfitting phenomenon,at the same time it can enhance the spatial discriminant of projection subspace.Then,the augmented Lagrange multiplier (ALM) is used to optimize the objective function.Finally,the nearest neighbor classifier is used for classification in subspace.We compared the related algorithms on AR,FERET face database,Stanford 40 Actions database,Caltech-UCSD Birds database and Oxford 102 Flowers database.The experimental results show that the proposed algorithm is effective.

Key words: Classification, Linear regression, Low rank, Regularization

中图分类号: 

  • TP391.41
[1]DONOHO D.High-dimensional data analysis:The curses and blessings of dimensionality[C]∥AMS Math Challenges Lecture.2000:1-32.
[2]YANG J,FRANGI A F,YANG J Y,et al.KPCA Plus LDA:a complete kernel Fisher discriminant frame work for feature extraction and recognition [J].IEEE Trans.Pattern Anal.Mach.Intell,2005,27(2):230-244.
[3]HE X F,YAN S C,HU Y X.Face recognition using Laplacian faces[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2005,27(3):328-340.
[4]TORRE F.A least-squares framework for component analysis [J].IEEE Trans.Pattern Anal.Mach.Intell.,2012,24(6),1041-1055.
[5]STRUTZ T.Data Fitting and Uncertainty:A Practical Introduction to Weighted Least Squares and Beyond[M].Wiesbaden,Germany:Vieweg and Teubner,2010.
[6]LI Y,NGOM A.Nonnegative least-squares methods for the classification of high-dimensional biological data [J].IEEE/ACM Transactions on Computational Biology & Bioinforma-tics,2013,10(2):447-456.
[7]XIANG S,NIE F,MENG G,et al.Discriminative least squares regression for multiclass classification and feature selection [J].IEEE Trans.Neural Netw.Learn.Syst.,2012,23(11):1738-1754.
[8]ZHANG Z,XU Y,YANG Y,et al.A survey of sparse representation:Algorithms and applications [J].IEEE Access,2015,3(1):490-530.
[9]WRIGHT J,YANG A Y,GANESH A,et al.Robust face recognition via sparse representation [J].IEEE Trans.Pattern Anal.Mach.Intell.,2009,31(2):210-227.
[10]ZHANG L,YANG M,FENG X.Sparse representation or collaborative representation:Which helps face recognition? [J].International Conference on Computer Vision,2012,2011(5):471-478.
[11]CAND S E J,LI X,MA Y,et al.Robust principal component analysis? [J].ACM,2011,58(3):11.
[12]CAI X,DING C,NIE F,et al.On the equivalent of low-rank li- near regressions and linear discriminant analysis based regressions [C]∥Proc.19th ACM SIGKDD Int.Conf.Knowl.Disco-very Data Mining,2013:1124-1132.
[13]ZHANG Z,LAI Z H,XU Y.Discriminative Elastic-Net Regularized Linear Regression [J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2017,26(3):1466-1481.
[14]WRIGHT J,PENG Y G,MA Y.Robust principal component analysis:exact recovery of corrupted low-rank mall-ices by convex optimization[J].International Conference on Neural Information Processing Systems,2009,58(3):2080-2088.
[15]LIN Z,CHEN M,MA Y.The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices[OL].https://arxiv.org/abs/1009.5055.
[16]NASEEM I,TOGNERI R,BENNAMOUN M.Linear regres- sion for face recognition [J].IEEE Trans.Pattern Anal.Mach.Intell.,2010,32(11):2106-2112.
[17]TROPP J A,GILBERT A C.Signal recovery from random measurements via orthogonal matching pursuit [J].IEEE Transactions on Information Theory,2007,53(12):4655-4666.
[18]OSBORNE M R,PRESNELL B,TURLACH B A.A new approach to variable selection in least squares problems [J].IMA Journal of Numerical Analysis,2000,20(3):389-403.
[19]XU Y,ZHANG D,YANG J.A two-phase test sample sparse representation method for use with face recognition [J].IEEE Transactions on Circuits and Systems for Video Technology,2011,21(9):1255-1262.
[20]MARTINEZ A,BENAVENTE R.The AR Face Database[R].Purdue University,Computer Vision Center,West Lafayette,1998.
[21]PHILLIPS P J.The facial recognition technology (FERET) database[OL].http:// www.itl.nist.gov/iad/humanid /feret/feret_master.html.
[22]YAO B,JIANG X,KHOSLA A,et al.Human action recognition by learning bases of action attributes and parts [C]∥International Conference on Computer Vision (ICCV).2011:1331-1338.
[23]WAH C,BRANSON S,WELINDER P,et al.The caltech-ucsd birds-200-2011 dataset[OL].http://www.vision.caltech.edu/visipedia/CUB-200-2011.html.
[24]NILSBACK M E,ZISSERMAN A.Automated flower classification over a large number of classes [C]∥Sixth Indian Confe-rence on Computer Vision,Graphics & Image Processing,2008(ICVGIP’08).2008:722-729.
[25]ABDI H,WILLIAMS L J.Principal component analysis[J].Wiley InterdiscipRev.Comput.Stat.,2010,2(4):433-459.
[26]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition [C]∥Proceedings of the International Conference on Learning Representations(ICLR).2014.
[1] 吕由, 吴文渊.
隐私保护线性回归方案与应用
Privacy-preserving Linear Regression Scheme and Its Application
计算机科学, 2022, 49(9): 318-325. https://doi.org/10.11896/jsjkx.220300190
[2] 陈志强, 韩萌, 李慕航, 武红鑫, 张喜龙.
数据流概念漂移处理方法研究综述
Survey of Concept Drift Handling Methods in Data Streams
计算机科学, 2022, 49(9): 14-32. https://doi.org/10.11896/jsjkx.210700112
[3] 周旭, 钱胜胜, 李章明, 方全, 徐常胜.
基于对偶变分多模态注意力网络的不完备社会事件分类方法
Dual Variational Multi-modal Attention Network for Incomplete Social Event Classification
计算机科学, 2022, 49(9): 132-138. https://doi.org/10.11896/jsjkx.220600022
[4] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[5] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[6] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[7] 武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航.
监督和半监督学习下的多标签分类综述
Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning
计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111
[8] 高振卓, 王志海, 刘海洋.
嵌入典型时间序列特征的随机Shapelet森林算法
Random Shapelet Forest Algorithm Embedded with Canonical Time Series Features
计算机科学, 2022, 49(7): 40-49. https://doi.org/10.11896/jsjkx.210700226
[9] 杨炳新, 郭艳蓉, 郝世杰, 洪日昌.
基于数据增广和模型集成策略的图神经网络在抑郁症识别上的应用
Application of Graph Neural Network Based on Data Augmentation and Model Ensemble in Depression Recognition
计算机科学, 2022, 49(7): 57-63. https://doi.org/10.11896/jsjkx.210800070
[10] 张洪博, 董力嘉, 潘玉彪, 萧宗志, 张惠臻, 杜吉祥.
视频理解中的动作质量评估方法综述
Survey on Action Quality Assessment Methods in Video Understanding
计算机科学, 2022, 49(7): 79-88. https://doi.org/10.11896/jsjkx.210600028
[11] 黄璞, 沈阳阳, 杜旭然, 杨章静.
基于局部约束特征线表示的人脸识别
Face Recognition Based on Locality Constrained Feature Line Representation
计算机科学, 2022, 49(6A): 429-433. https://doi.org/10.11896/jsjkx.210300169
[12] 杨涵, 万游, 蔡洁萱, 方铭宇, 吴卓超, 金扬, 钱伟行.
基于步态分类辅助的虚拟IMU的行人导航方法
Pedestrian Navigation Method Based on Virtual Inertial Measurement Unit Assisted by GaitClassification
计算机科学, 2022, 49(6A): 759-763. https://doi.org/10.11896/jsjkx.211200148
[13] 杜丽君, 唐玺璐, 周娇, 陈玉兰, 程建.
基于注意力机制和多任务学习的阿尔茨海默症分类
Alzheimer's Disease Classification Method Based on Attention Mechanism and Multi-task Learning
计算机科学, 2022, 49(6A): 60-65. https://doi.org/10.11896/jsjkx.201200072
[14] 李小伟, 舒辉, 光焱, 翟懿, 杨资集.
自然语言处理在简历分析中的应用研究综述
Survey of the Application of Natural Language Processing for Resume Analysis
计算机科学, 2022, 49(6A): 66-73. https://doi.org/10.11896/jsjkx.210600134
[15] 邓凯, 杨频, 李益洲, 杨星, 曾凡瑞, 张振毓.
一种可快速迁移的领域知识图谱构建方法
Fast and Transmissible Domain Knowledge Graph Construction Method
计算机科学, 2022, 49(6A): 100-108. https://doi.org/10.11896/jsjkx.210900018
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!