计算机科学

计算机科学 ›› 2017, Vol. 44 ›› Issue (7): 185-190.doi: 10.11896/j.issn.1002-137X.2017.07.033

• 人工智能 • 上一篇    下一篇

一种自适应的多类Boosting分类算法

王世勋,潘鹏,陈灯,卢炎生   

  1. 河南师范大学计算机与信息工程学院 新乡453007,华中科技大学计算机科学与技术学院 武汉430074,武汉工程大学智能机器人湖北省重点实验室 武汉430205,华中科技大学计算机科学与技术学院 武汉430074
  • 出版日期:2018-11-13 发布日期:2018-11-13
  • 基金资助:
    本文受河南省自然科学基金(162300410177),河南省高等学校重点科研项目(17A520040),河南师范大学博士科研启动基金(qd15134)资助

Adaptive Multiclass Boosting Classification Algorithm

WANG Shi-xun, PAN Peng, CHEN Deng and LU Yan-sheng   

  • Online:2018-11-13 Published:2018-11-13

PDF (PC)

724

摘要： 许多实际问题涉及到多分类技术,该技术能有效地缩小用户与计算机之间的理解差异。在传统的多类Boosting方法中,多类损耗函数未必具有猜测背离性,并且多类弱学习器的结合被限制为线性的加权和。为了获得高精度的最终分类器,多类损耗函数应具有多类边缘极大化、贝叶斯一致性与猜测背离性。此外,弱学习器的缺点可能会限制线性分类器的性能,但它们的非线性结合可以提供较强的判别力。根据这两个观点,设计了一个自适应的多类Boosting分类器,即SOHPBoost算法。在每次迭代中,SOHPBoost算法能够利用向量加法或Hadamard乘积来集成最优的多类弱学习器。这个自适应的过程可以产生多类弱学习的Hadamard乘积向量和,进而挖掘出数据集的隐藏结构。实验结果表明,SOHPBoost算法可以产生较好的多分类性能。

关键词: 多类Boosting,损耗函数,猜测背离性,非线性结合

Abstract: Many practical problems have involved classification technology which can effectively reduce the comprehension difference between users and computers.In the traditional multiclass Boosting methods,multiclass loss function does not necessarily have guess-aversion,and the combinations of multiclass weak learners are confined to linear weighted sum.To get a final classifier with high accuracy,multiclass loss function need contain three main properties,namely margin maximization,Bayes consistency and guess-aversion.Moreover,the weakness of weak learners may limit the performance of linear classifier,but their nonlinear combinations should provide strong discrimination.Therefore,we designed an adaptive multiclass Boosting classifier,namely SOHPBoost algorithm.At every iteration,our algorithm can add the best weak learner to the current ensemble according to vector addition or Hadamard product.This adaptive process can create the sum of Hadamard products of weak learners,and mine the hidden structure of dataset.The experi-ments show that SOHPBoost algorithm can provide more advantageous performance of multiclass classification.

Key words: Multiclass Boosting,Loss function,Guess-aversion,Nonlinear combination

[1] FREUND Y,SCHAPIRE R E.A Decision-theoretic Generalization of On-line Learning and an Application to Boosting[J].Journal of Computer and System Sciences,1997,55(1):119-139.
[2] FRIEDMAN J,HASTIE T,TIBSHIRANI R.Additive Logistic Regression:A Statistical View of Boosting[J].The Annals of Statistics,2000,28(2):337-407.
[3] MASNADI-SHIRAZI H,VASCONCELOS N.On the Design of Loss Functions for Classification:Theory,Robustness to Out-liers,and Savagebooost[C]∥Proceedings of Advances in Neural Information Processing Systems.2009:1049-1056.
[4] MASNADI-SHIRAZI H,MAHADEVAN V, VASCONCELOS N.On the Design of Robust Classifiers for Computer Vision[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.2010:779-786.
[5] SABERIAN M J,MASNADI-SHIRAZI H,VASCONCELOS N.TaylorBoost:First and Second-order Boosting Algorithms with Explicit Margin Control[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.2011:2929-2934.
[6] ABOUELENIEN M,YUAN X.Boosting for Learning fromMulticlass Data Sets via a Regularized Loss Function[C]∥Proceedings of IEEE International Conference on Granular Computing.2013:4-9.
[7] DANIELSSON O,RASOLZADEH B,CARLSSON S.GatedClassifiers:Boosting under High Intra-Class Variation[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.2011:2673-2680.
[8] SABERIAN M J,VASCONCELOS N.Boosting Algorithms for Simultaneous Feature Extraction and Selection[C]∥Procee-dings of IEEE Conference on Computer Vision and Pattern Re-cognition.2012:2448-2455.
[9] DIETTERICH T G,BAKIRI G.Solving Multiclass Learning Pro-blems via Error Correcting Output Codes[J].Journal of Artifical Intelligence Research,1995,2:263-286 .
[10] SEIFFERT C,KHOSHGOFTAAR T M, HULSE V,et al.RUSBoost:Improving Classification Performance When Trai-ning Data Is Skewed[C]∥Proceedings of IEEE International Conference on Pattern Recognition.2008:1-4.
[11] KGL B,BUSA-FEKETE R.Boosting Products of Base Classifiers[C]∥Proceedings of the 26th International Conference on Machine Learning.2009:497-504.
[12] ZHU J,ZOU H,ROSSET S,et al.Multi-class AdaBoost[J].Satistics and Its Interface,2009,2(3):349-360.
[13] MUKHERJEE I,SCHAPIRE R E.A Theory of MulticlassBoosting[J].Journal of Machine Learning Research,2013,14(1):437-497.
[14] SABERIAN M J,VASCONCELOS N.Multiclass Boosting:Theory and Algorithms[C]∥Proceedings of Advances in Neural Information Processing Systems.2011:2124-2132.
[15] MASNADI-SHIRAZI H,VASCONCELOS N.Cost-sensitive Boos-ting[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(2):294-309.
[16] ABE N,ZADROZNY B,LANGFORD J.An Iterative Methodfor Multi-class Cost-sensitive Learning[C]∥Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2004:3-11.
[17] LOZANO A C,ABE N.Multi-class Cost-sensitive Boosting with P-norm Loss Functions[C]∥Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2008:506-514.
[18] WANG J.Boosting the Generalized Margin in Costsensitive Mul-ticlass Classification[J].Journal of Computational and Graphical Statistics,2013,22(1):178-192.
[19] BEIJBOM O,SABERIAN M,KRIEGMAN D,et al.Guess-Averse Loss Functions for Cost-sensitive Multiclass Boosting[C]∥Proceedings of the 31st International Conference on Machine Lear-ning.2014:586-594.
[20] UCI:Machine Learning Repositery.http://archive.ics.uci.edu/ml/datasets.html.
[21] WARMUTH M K,LIAO J,RTSCH G.Totally CorrectiveBoosting Algorithm That Maximize the Margin[C]∥Procee-dings of 23rd ACM International Conference on Machine Lear-ning.2006:1001-1008.
No related articles found!
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发