Computer Science ›› 2017, Vol. 44 ›› Issue (12): 42-47.doi: 10.11896/j.issn.1002-137X.2017.12.008

Previous Articles     Next Articles

SVRRPMCC:A Regularization Path Approximation Algorithm of Support Vector Regression

WANG Mei, WANG Sha-sha, SUN Ying-qi, SONG Kao-ping, TIAN Feng and LIAO Shi-zhong   

  • Online:2018-12-01 Published:2018-12-01

Abstract: The regularization path algorithm is an efficient method for numerical solution to the support vector regression (SVR) problem, which can obtain all possible values of regularization parameters and the solutions of SVR in the time complexity equivalent to a SVR solution.Existing SVR regularization path algorithms include solving a system of iteration equations.The existing accurate approaches are difficult to apply to large-scale problems.Recently,there has been many interests about the approximation approach.And a new approximation algorithm for SVR regularization path named SVRRPMCC was proposed in this paper.Firstly,SVRRPMCC applied Monte Carlo method to randomly sample the coefficient matrix of the system of iteration equations.Then it used the Cholesky factorization method to obtain the coefficient inverse matrix.Further,the error bound and the computational complexity about the algorithm SVRRPMCC were analyzed.Experimental results on benchmark datasets it used show the validity and efficiency of the SVRRPMCC.

Key words: Support vector regression,Regularization path,Matrix approximation,Monte Carlo sample,Cholesky decomposition

[1] CHENG X Q,JIN X L,et al.Survey on Big Data System and Analytic Technology[J].Journal of Software,2014,5(9):1889-1908.(in Chinese) 程学旗,靳小龙,等.大数据系统和分析技术综述[J].软件学报,2014,5(9):1889-1908.
[2] HUANG Y H.Research Progress on Big Data Machine Lear-ning System[J].Big Data Research,2015,1:1-20.(in Chinese) 黄宜华.大数据机器学习系统研究进展[J].大数据,2015,1:1-20.
[3] HE Q,LI N,et al.A Survey of Machine Learning Algorithms for Big Data[J].Pattern Recognition and Artificial Intelligence,2014,7(4):327-335.(in Chinese) 何清,李宁,等.大数据下的机器学习算法综述[J].模式识别与人工智能,2014,7(4):327-335.
[4] EFRON B,HASTIE T,JOHNSTONE I,et al.Least angle regression [J].The Annals of Statistics,2004,2(2):407-499.
[5] HASTIE T,ROSSET S,Tibshirani R,et al.The entire regularization path for the support vector machine [J].Journal of Machine Learning Research,2004,5:1391-1415.
[6] GUNTER L,ZHU J.Efficient computation and model selection for the support vector regression [J].Neural Computation,2007,9(6):1633-1655.
[7] ROSSET S,ZHU J.Piecewise linear regularized solution paths [J].The Annals of Statistics,2007,5(3):1012-1030.
[8] KARASUYAMA M,TAKEUCHI I.Suboptimal solution path algorithm for support vector machine [C]∥Proc.of the 28th Int.Conf.on Machine Learning.New York:Association for Computing Machinery,2011.
[9] ONG C J,SHAO S Y,YANG J B.An improved algorithm for the solution of the regularization path of support vector machine [J].IEEE Transactions on Neural Networks,2010,1(3):451-462.
[10] LIAO S Z,WANG M,ZHAO Z H.Regularization Path Algorithm of SVM via Positive Definite Matrix[J].Journal of Computer Research and Development,2013,0(11):2253-2261.(in Chinese) 廖士中,王梅,赵志辉.正定矩阵支持向量机正则化路径算法[J].计算机研究与发展,2013,0(11):2253-2261.
[11] DING L Z,LIAO S Z.Approximate model selection on regularization path for support vector machines[J].Journal of Computer Research and Development,2012,9(6):1248-1255.(in Chinese) 丁立中,廖士中.基于正则化路径的支持向量机近似模型选择[J].计算机研究与发展,2012,9(6):1248-1255.
[12] WANG L F,SHEN X T.Multi-category support vector ma-chines,feature selection and solution path [J].Statistica Sinica,2006,6(2):617-633.
[13] RAKOTOMAMONJY A,DAVY M.One-class SVM regularization path and comparison with alpha seeding [C]∥Proc.of European Symp.on Artificial Neural Networks.Belgium:D-Facto Publications,2007:271-276.
[14] ZAPIEN K,GRTNER T,GASSO G,et al.Regularisation path for ranking SVM [C]∥Proc.of European Symp.on Artificial Neural Networks.Belgium:D-Facto Publications,2008:415-420.
[15] KARASUYAMA M,TAKEUCHI I.Nonlinear regularizationpath for quadratic loss support vector machines [J].IEEE Transactions on Neural Networks,2011,2(10):1613-1625.
[16] LOOSLI G,GASSO G,CANU S.Regularization paths forv-SVM and v-SVR [C]∥Proc of the 4th International Sympo-sium on Neural Network.Heidelberg:Springer,2007:486-496.
[17] GASSO G,ZAPIEN K,CANU S.Sparsity regularization path for semi-supervised SVM [C]∥The 6th Int Conf on Machine Learning and Applications.Los Alamitos,CA:IEEE Computer Society,2007:25-30.
[18] WANG G,WANG F,CHEN T,et al.Solution path for manifold regularized semisupervised classification [J].IEEE Transactions on Systems,Man,and Cybernetics,Part B:Cybernetics,2012,42(2):308-319.
[19] BACH F R,THIBAUX R,JORDAN M I.Computing regularization paths for learning multiple kernels [M]∥Advances in Neural Information Processing Systems 17.Cambridge,MA:MIT Press,2004:41-48.
[20] BLOMBERG N,ROJAS C R,BO W.Approximate Regularization Path for Nuclear Norm Based H2 Model Reduction[J].Decision & Control,2015,5:3637-3641.
[21] WANG M.Model Combination of SVM on Regularization Path [D].Tianjin:Tianjin University,2013.(in Chinese) 王梅.正则化路径上支持向量机模型组合[D].天津:天津大学,2013.
[22] HASTIE T,TIBSHIRANI R,FRIEDMAN J H.The Elements of Statistical Learning:Data Mining,Inference,and Prediction [M].New York:Springer,2001.
[23] BERROUBB C,GLAVIEUX A,THITIMASHIMA P.NearShannon limit error-correcting coding and decoding:Turbo codes[C]∥IEEE Int Conf on Communications.NewYork:IEEE,1993:1064-1071.
[24] WEI C J,ZHANG C S,LIU J.A new method of solving inverse matrix based on Cholesky matrix [J].Electronic Design Engineering,2014,2(1):159-164.(in Chinese) 魏婵娟,张春水,刘健.一种基于Cholesky分解的快速矩阵求逆方法设计[J].电子设计工程,2014,2(1):159-164.
[25] HORN R,JOHNSON C.Matrix Analysis [M].Cambridge,UK:Cambridge University Press,1985.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!