计算机科学 ›› 2016, Vol. 43 ›› Issue (3): 49-53.doi: 10.11896/j.issn.1002-137X.2016.03.009

• 第十五届中国机器学习会议 • 上一篇    下一篇

基于马氏距离的孪生多分类支持向量机

张谢锴,丁世飞   

  1. 中国矿业大学计算机科学与技术学院 徐州221116,中国矿业大学计算机科学与技术学院 徐州221116;中国科学院计算技术研究所智能信息处理重点实验室 北京100190
  • 出版日期:2018-12-01 发布日期:2018-12-01
  • 基金资助:
    本文受国家重点基础研究发展规划(973计划)(2013CB329502),国家自然科学基金(61379101)资助

Mahalanobis Distance-based Twin Multi-class Classification Support Vector Machine

ZHANG Xie-kai and DING Shi-fei   

  • Online:2018-12-01 Published:2018-12-01

摘要: 孪生支持向量机(TWSVM)的研究是近来机器学习领域的一个热点。TWSVM具有分类精度高、训练速度快等优点,但训练时没有充分利用样本的统计信息。作为TWSVM的改进算法,基于马氏距离的孪生支持向量机(TMSVM)在分类过程中考虑了各类样本的协方差信息,在许多实际问题中有着很好的应用效果。然而TMSVM的训练速度有待提高,并且仅适用于二分类问题。针对这两个问题,将最小二乘思想引入TMSVM,用等式约束取代TMSVM中的不等式约束,将二次规划问题的求解简化为求解两个线性方程组,得到基于马氏距离的最小二乘孪生支持向量机(LSTMSVM),并结合有向无环图策略(DAG)设计出基于马氏距离的最小二乘孪生多分类支持向量机。为了减少DAG结构的误差累积,构造了基于马氏距离的类间可分性度量。人工数据集和UCI数据集上的实验均表明,所提算法不仅有效,而且相对于传统多分类SVM,其分类性能有明显提高。

关键词: 孪生支持向量机,马氏距离,多分类,有向无环图

Abstract: Twin support vector machine (TWSVM) is a recent hot spot in the field of machine learning.TWSVM achieves fast training speed and good performance for data classification.However,it does not take full advantage of the statistical information in training samples.As an improved algorithm of TWSVM,mahalanobis distance-based twin support vector machine (TMSVM) takes the covariance of each class of data into consideration and is suitable for many real problems.However,the learning speed of TMSVW remains to be improved.Moreover,the approach only deals with binary classification problems.To solve these problems,this paper formulated a least squares version of TMSVM by considering equality type constraints instead of inequalities of TMSVM,which makes the solution follow from solving a set of linear equations instead of quadratic programming,and extended the binary classier into a new multiclass classification algorithm with directed acyclic graph (DAG).In order to reduce error accumulation in DAG structure,a mahalanobis distance-based distance measure was designed as the class separability criterion.The experimental results on toy dataset and UCI datasets show that the proposed algorithm is efficient and has better classification accuracy than traditional multi-class SVM.

Key words: Twin support vector machine,Mahalanobis distance,Multi-class classification,Directed acyclic graph

[1] Cortes C,Vapnik V N.Support vector networks[J].MachinesLearning,1995,20(2):273-297
[2] Vapnik V N.The nature of statistical learning theory[M].New York:Springer,1998
[3] Mangasarian O L,Wild E.Multisurface proximal support vector machine classification via generalized eigenvalues[J].IEEE Transactions on Pattern Aalysis and Machine Intelligence,2006,28(1):69-74
[4] Jayadeva,Khemchandni R,Chandra S.Twin support vector machines for pattern classification[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(5):905-910
[5] Ding Shie-fei,Yu Jun-zhao,Qi Bing-juan,et al.An overview on twin support vector machines[J].Artificial Intelligence Review,2014,42(2):245-252
[6] Kumar M A,Gopal M.Least squares twin support vector ma-chines for pattern classification[J].Expert Systems with Applications,2009,36(4):7535-7543
[7] Shao Yuan-hai,Chen Wei-jie,Zhang Jing-jing,et al.An efficient weighted Lagrangian twin support vector machine for imba-lanced data classification[J].Pattern Recognition,2014,47(9):3158-3167
[8] Wang Di,Ye Qiao-lin,Ye Ning.Localized multi-plane TWSVM classifier via manifold regularization[C]∥2010 Second International Conference on Intelligent Human-Machine Systems and Cybernetics,2011.New York:IEEE,2010:70-73
[9] Wang Ya-nan,Zhao Xi,Tian Ying-jie.Local and global regula-rized twin SVM[J].Procedia Computer Science,2013,18:1710-1719
[10] Kumar M A,Gopal M.Application of smoothing technique on twin support vector machines[J].Pattern Recognition Letters,2008,29(13):1842-1848
[11] Ding Shi-fei,Huang Hua-juan,Yu Jun-zhao,et al.Polynomial smooth twin support vector machines based on invasive weed optimization algorithm[J].Journal of Computers,2014,9(5):2063-2071
[12] Ding Shi-fei,Huang Hua-juan,Shi Zhong-zhi.Weighted smooth CHKS twin support vector machines[J].Journal of Software,2013,4(11):2548-2557(in Chinese) 丁世飞,黄华娟,史忠植.加权光滑CHKS孪生支持向量机[J].软件学报,2013,24(11):2548-2557
[13] Ding Shi-fei,Wu Fu-lin,Shi Zhog-zhi.Wavelet twin support vector machine[J].Neural Computing and Applications,2014,25(6):1241-1247
[14] Xie Xi-jiong,Sun Shi-liang.Multitask centroid twin support vector machines[J].Neurocomputing,2015,149(2):1085-1091
[15] Peng Xin-jun,Xu Dong.Twin Mahalanobis distance-based support vector machines for pattern recognition[J].Information Sciences,2012,200:22-37
[16] Chu Mao-xiang,Wang An-na,Gong Rong-fei,et al.Multi-class Classification Methods of Enhanced LS-TWSVM for Strip Steel Surface Defects[J].Journal of Iron and Steel Research International,2014,21(2):174-180
[17] Xu Yi-tian,Guo Rui.A twin hyper-sphere multi-class classification support vector machine[J].Journal of Intelligent and Fuzzy Systems,2014,27(4):1783-1790
[18] Yang Zhi-xia,Shao Yuan-hai,Zhang Xiang-sun.Multiple birth support vector machine for multi-class classification[J].Neural Computing and Applications,2013,22(1):153-161
[19] Chen J,Ji G R.Multi-class LSTSVM classifier based on optimal directed acyclic graph[C]∥The 2nd International Conference on Computer and Automation Engineering,2010.New York:IEEE,2010:100-104
[20] Xu Yi-tian,Guo Rui,Wang Lai-sheng.A twin multi-class classification support vector machine[J].Cognitive Computation,2013,5(4):580-588
[21] Xie Jun-ying,Hone Ka-ta,Xie Wei-xin,et al.Extending twinsupport vector machine classifier for multi-category classification problems[J].Intelligent Data Analysis,2013,17(4):649-664
[22] Shao Yuan-hai,Chen Wei-jie,Huang Wen-biao,et al.The best separating decision tree twin support vector machine for multi-class classification[J].Procedia Computer Science,2013,17:1032-1038
[23] Wang Xue-min.Applied multivariate analysis(4th edition)[M].Shanghai:Shanghai University of Finance and Economics Press,2014:25-34(in Chinese) 王学民.应用多元分析(第4版)[M].上海:上海财经大学出版社,2014:25-34

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!