Computer Science ›› 2017, Vol. 44 ›› Issue (7): 185-190.doi: 10.11896/j.issn.1002-137X.2017.07.033

Previous Articles     Next Articles

Adaptive Multiclass Boosting Classification Algorithm

WANG Shi-xun, PAN Peng, CHEN Deng and LU Yan-sheng   

  • Online:2018-11-13 Published:2018-11-13

Abstract: Many practical problems have involved classification technology which can effectively reduce the comprehension difference between users and computers.In the traditional multiclass Boosting methods,multiclass loss function does not necessarily have guess-aversion,and the combinations of multiclass weak learners are confined to linear weighted sum.To get a final classifier with high accuracy,multiclass loss function need contain three main properties,namely margin maximization,Bayes consistency and guess-aversion.Moreover,the weakness of weak learners may limit the performance of linear classifier,but their nonlinear combinations should provide strong discrimination.Therefore,we designed an adaptive multiclass Boosting classifier,namely SOHPBoost algorithm.At every iteration,our algorithm can add the best weak learner to the current ensemble according to vector addition or Hadamard product.This adaptive process can create the sum of Hadamard products of weak learners,and mine the hidden structure of dataset.The experi-ments show that SOHPBoost algorithm can provide more advantageous performance of multiclass classification.

Key words: Multiclass Boosting,Loss function,Guess-aversion,Nonlinear combination

[1] FREUND Y,SCHAPIRE R E.A Decision-theoretic Generalization of On-line Learning and an Application to Boosting[J].Journal of Computer and System Sciences,1997,55(1):119-139.
[2] FRIEDMAN J,HASTIE T,TIBSHIRANI R.Additive Logistic Regression:A Statistical View of Boosting[J].The Annals of Statistics,2000,28(2):337-407.
[3] MASNADI-SHIRAZI H,VASCONCELOS N.On the Design of Loss Functions for Classification:Theory,Robustness to Out-liers,and Savagebooost[C]∥Proceedings of Advances in Neural Information Processing Systems.2009:1049-1056.
[4] MASNADI-SHIRAZI H,MAHADEVAN V, VASCONCELOS N.On the Design of Robust Classifiers for Computer Vision[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.2010:779-786.
[5] SABERIAN M J,MASNADI-SHIRAZI H,VASCONCELOS N.TaylorBoost:First and Second-order Boosting Algorithms with Explicit Margin Control[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.2011:2929-2934.
[6] ABOUELENIEN M,YUAN X.Boosting for Learning fromMulticlass Data Sets via a Regularized Loss Function[C]∥Proceedings of IEEE International Conference on Granular Computing.2013:4-9.
[7] DANIELSSON O,RASOLZADEH B,CARLSSON S.GatedClassifiers:Boosting under High Intra-Class Variation[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.2011:2673-2680.
[8] SABERIAN M J,VASCONCELOS N.Boosting Algorithms for Simultaneous Feature Extraction and Selection[C]∥Procee-dings of IEEE Conference on Computer Vision and Pattern Re-cognition.2012:2448-2455.
[9] DIETTERICH T G,BAKIRI G.Solving Multiclass Learning Pro-blems via Error Correcting Output Codes[J].Journal of Artifical Intelligence Research,1995,2:263-286 .
[10] SEIFFERT C,KHOSHGOFTAAR T M, HULSE V,et al.RUSBoost:Improving Classification Performance When Trai-ning Data Is Skewed[C]∥Proceedings of IEEE International Conference on Pattern Recognition.2008:1-4.
[11] KGL B,BUSA-FEKETE R.Boosting Products of Base Classifiers[C]∥Proceedings of the 26th International Conference on Machine Learning.2009:497-504.
[12] ZHU J,ZOU H,ROSSET S,et al.Multi-class AdaBoost[J].Satistics and Its Interface,2009,2(3):349-360.
[13] MUKHERJEE I,SCHAPIRE R E.A Theory of MulticlassBoosting[J].Journal of Machine Learning Research,2013,14(1):437-497.
[14] SABERIAN M J,VASCONCELOS N.Multiclass Boosting:Theory and Algorithms[C]∥Proceedings of Advances in Neural Information Processing Systems.2011:2124-2132.
[15] MASNADI-SHIRAZI H,VASCONCELOS N.Cost-sensitive Boos-ting[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(2):294-309.
[16] ABE N,ZADROZNY B,LANGFORD J.An Iterative Methodfor Multi-class Cost-sensitive Learning[C]∥Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2004:3-11.
[17] LOZANO A C,ABE N.Multi-class Cost-sensitive Boosting with P-norm Loss Functions[C]∥Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2008:506-514.
[18] WANG J.Boosting the Generalized Margin in Costsensitive Mul-ticlass Classification[J].Journal of Computational and Graphical Statistics,2013,22(1):178-192.
[19] BEIJBOM O,SABERIAN M,KRIEGMAN D,et al.Guess-Averse Loss Functions for Cost-sensitive Multiclass Boosting[C]∥Proceedings of the 31st International Conference on Machine Lear-ning.2014:586-594.
[20] UCI:Machine Learning Repositery.http://archive.ics.uci.edu/ml/datasets.html.
[21] WARMUTH M K,LIAO J,RTSCH G.Totally CorrectiveBoosting Algorithm That Maximize the Margin[C]∥Procee-dings of 23rd ACM International Conference on Machine Lear-ning.2006:1001-1008.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!