Computer Science ›› 2016, Vol. 43 ›› Issue (12): 139-145.doi: 10.11896/j.issn.1002-137X.2016.12.025

Previous Articles     Next Articles

Novel Multi-scale Kernel SVM Method Based on Sample Weighting

SHEN Jian, JIANG Yun, ZHANG Ya-nan and HU Xue-wei   

  • Online:2018-12-01 Published:2018-12-01

Abstract: Multi-kernel learning has been a new research focus in the current kernel machine learning field.Through mapping data into high dimensional space,kernel methods increase the computational power of linear classifier and they are an effective way to solve the problem of nonlinear model analysis and classification.In some complex situations,ne-vertheless,the kernel learning method of single kernel function can not completely satisfy the requirements of heterogeneous data or irregular data as well as samples of large size and non-flat distribution.Therefore,it is necessary to deve-lop multiple kernel functions in order to get better results.In this paper,we proposed a new SVM method for multi-scale kernel learning based on sample weighting,which is assigned via fitting abilities of distinct scales kernel functions for samples.Through the experimental analysis on several data sets,we can get that the method proposed in this paper can attain better classification accuracy on each data set.

Key words: Multi-kernel learning,Map,Nonlinear model,Heterogeneous data

[1] Boser B E,Guyon I M,Vapnik V N.A training algorithm for optimal margin classifiers[M]∥Haussler D.eds.,5th Annual ACM Workshop on COLT,Pittsburgh,PA.ACM Press.,1992
[2] Vapnik V.The nature of statistical learning theory[M].New York:Springer-Verlag,1995
[3] Wang L P.Support vector machine:theory and application [M].Berlin:Springer-Verlag,2005
[4] Schoelkopf B,Smola A,Muller K R.Nonlinear component analysis as a kernel eigenvalue problem[J].Neural Computation,1998,10(5):1299-1319
[5] Scholkopf B,Mika S,Burges C J C,et al.Input space versus feature space in kernel-based methods[J].IEEE Transactions on Neural Network,1999,10(5):1000-1017
[6] Muller K R,Mika S,Ratsch G,et al.An introduction to kernel based learning algorithms[J].IEEE Transactions on Neural Networks,2001,12(2):181-201
[7] Mercer J.Functions of positive and negative type and their connection with the theory of integral equations[J].Philosophical Transactions of the Royal Society of London,Series A,1909,209:415-446
[8] Aronszajn N.Theory of reproducing kernels[J].Transactions of the American Mathematical Society,1950,68(3):337-404
[9] Aizerman A,Braverman E M,Rozoner L I.Theoretical foundations of the potential function method in pattern recognition learning[J].Automation and Remote Control,1964,25(5):821-837
[10] Smola A J,Scholkopf B.A tutorial on support vector regression[J].Statistics and Computing,2004,14(3):199-222
[11] Burges C J C.A tutorial on support vector machines for pattern recognition[J].Data Mining and Knowledge Discovery,1998,2(2):121-167
[12] Kerm P V.Adaptive kernel density estimation[J].Stata Jour-nal,2003,3(2):148-156
[13] Scholkopf B,Mika S,Smola A,et al.Kernel PCA pattern reconstruction via approximation preimages[C]∥Proceedings of the International Conference of Artificial Neural Networks.Skovde,Sweden:IEEE,1998:147-152
[14] Mika S,Ratsch G,Weston J,et al.Fisher discriminant analysis with kernels[C]∥Proceedingsof the Conference on Neural Networks for Signal Processing.Washington D.C.,USA:IEEE,1999:41-48
[15] Baudat G,Anouar F.Generalized discriminant analysis using a kernel approach[J].Neural Computation,2000,12(10):2385-2404
[16] Wang Hong-qiao,Sun Fu-chun,Cai Yan-ning,et al.On multiple kernel learning methods[J].Acta Automatic Sinica,2010,36(8):1037-1050(in Chinese) 汪洪桥,孙富春,蔡艳宁,等.多核学习方法[J].自动化学报,2010,6(8):1037-1050
[17] Huang C,Chen Y,Chen W,et al.Gastroesophageal Reflux Disea-se Diagnosis Using Hierarchical Heterogeneous Descriptor Fusion Support Vector Machine[C]∥IEEE Engineering in Medicine and Biology Society.2015:1-10
[18] Penga S,Hua Qing-hua,Chen Yin-li,et al.Interactive,Improved support vector machine algorithm for heterogeneous data [J].Pattern Recognition,2015,48(6):2072-2083
[19] Sonnenburg S,Ratsch G,Schafer C,et al.Large scale multiple kernel learning[J].The Journal of Machine Learning Research,2006,7(7):1531-1565
[20] Xiao Yu-lin,Zhong Shang-ping.An improved online multiplekernel classification algorithm based on double updating online learning [C]∥2014 International Conference on Cloud Computing and Internet of Things.2014:109-113
[21] Bach F R.Consistency of the group Lasso and multiple kernel learning[J].The Journal of Machine Learning Research,2008,9(6):1179-1225
[22] Cortes C,Mohri M,Rostamizadeh A.Learning sequence kernels[C]∥Proceedings of the International Conference on Machine Learning for Signal Processing.Washington D.C.,USA:IEEE,2008:2-8
[23] Rakotomamonjy A,Bach F R,Canu S,et al.Simple MKL[J].The Journal of Machine Learning Research,2008,9(11):2491-2521
[24] Zheng D N,Wang J X,Zhao Y N.Nonflat function estimation with a multi-scale support vector regession[J].Neurocompu-ting,2006,70(1-3):420-429
[25] Wang Jing-yan,Bensmailc H,Gao Xin.Feature selection andmulti-kernel learning for sparse representation on a manifold [J].Neural Networks,2014,51(3):9-16
[26] Xua Lin,Feng Yan-qiu,Liu Xiao-yun,et al.Robust GRAPPAreconstruction using sparse multi-kernel learning with least squares support vector regression [J].Magnetic Resonance Imaging,2014,2(1):91-101
[27] Wang Jing-yan,Huang Jian-hua,Sun Yi-jun,et al.Feature selection and multi-kernel learning for adaptive graph regularized nonnegative matrix factorization[J].Expert Systems with Applications,2015,2(3):1278-1286
[28] Lanckriet G R G,Cristianini N,Bartlett P,et al.Learning the kernel matrix with semi definite programming[J].The Journal of Machine Learning Research,2004,5(1):27-72
[29] Lee W J,Verzakov S,Duin R P.Kernel combination versus classifier combination[C]∥Proceedings of the 7th International Workshop on Multiple Classifier Systems.Prague,Czech Republic:Springer,2007:22-31
[30] Bi J B,Zhang T,Bennett K P.Column-generation boostingmethods for mixture of kernels[C]∥Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Disco-very and Data Mining.Seattle,USA:ACM,2004:521-526
[31] Kingsbury N,Tay D B H,Palaniswami M.Multi-Scale Kernel Me-thods for Classification[C]∥Proc.of the IEEE Workshop on Machine Learning for Signal Processing.Mystic,USA,2005:43-48
[32] Yang Zhen,Guo Jun,Xu Wei-ran.Multi-Scale Support Vector Machine for Regression Estimation[C]∥ Proc.of the 3rd International Symposiumon Neural Networks.Chengdu,China,2006:1030-1037
[33] Izmailov R,Bassu D,McIntosh A,et al.Application of multi-scale singular vector decomposition to vessel classification in overhead satellite imagery[C]∥Seventh International Confe-rence on Digital Image Processing (ICDIP 2015).2015
[34] Vapnik V N.统计学习理论[M].许建华,张学工,译.北京:电子工业出版社,2009
[35] Ding Shi-fei,Qi Bing-juan,Tan Hong-yan.An overview on theory and algorithm of support vector machines [J].Journal of University of Electronic Science and Technology of China,2011,0(1):2-10(in Chinese) 丁世飞,齐丙娟,谭红艳.支持向量机理论与算法研究综述[J].电子科技大学学报,2011,0(1):2-10
[36] Platt J C,Cristianini N,Shawe-Taylor J.Large Margin DAGs for multiclass classification[J].Advances in Neural Information Processing Systems,2000,12(3):547-553
[37] Manivannan K,Aggarwal P,Devabhaktuni V,et al.Particulate matter characterization by gray level co-occurrence matrix based support vector machines[J].Journal of hazardous materials,2012,223-224(2):94-103
[38] Hsu C W,Lin C J.A Comparison of Methods for MulticlassSupport Vector Machines [J].IEEE Transactions on Neural Networks,2002,3(3):415-425
[39] Kre U,et al.Pairwise classification and support vector machines[C]∥Advances in Kernel Methods.MIT Press,1999:255-268
[40] Larochelle H,Bengio Y.Classification using Discriminative Restricted Boltzmann Machines[C]∥Proceedings of the 25th International Conference on Machine Learning,2008.Helsinki,Finland,2008:1-8

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!