Volume 2 (2009) 349–360 | Ji Zhu†‡, Hui Zou§, Saharon Rosset and Trevor Hastie
This paper introduces a new algorithm, Multi-class AdaBoost (SAMME), which extends the AdaBoost algorithm to multi-class classification problems without reducing it to multiple two-class problems. The authors show that SAMME is equivalent to a forward stagewise additive modeling algorithm that minimizes a novel exponential loss function for multi-class classification. This exponential loss function is shown to be a member of a class of Fisher-consistent loss functions for multi-class classification. The paper also provides theoretical justification for the use of the multi-class exponential loss and demonstrates the effectiveness of SAMME through numerical experiments on both simulation data and real-world datasets. The results indicate that SAMME performs well and is comparable to other popular multi-class classification algorithms, such as AdaBoost.MH. The authors conclude that SAMME is a natural extension of the AdaBoost algorithm to multi-class problems and highlights its advantages in terms of simplicity and performance.This paper introduces a new algorithm, Multi-class AdaBoost (SAMME), which extends the AdaBoost algorithm to multi-class classification problems without reducing it to multiple two-class problems. The authors show that SAMME is equivalent to a forward stagewise additive modeling algorithm that minimizes a novel exponential loss function for multi-class classification. This exponential loss function is shown to be a member of a class of Fisher-consistent loss functions for multi-class classification. The paper also provides theoretical justification for the use of the multi-class exponential loss and demonstrates the effectiveness of SAMME through numerical experiments on both simulation data and real-world datasets. The results indicate that SAMME performs well and is comparable to other popular multi-class classification algorithms, such as AdaBoost.MH. The authors conclude that SAMME is a natural extension of the AdaBoost algorithm to multi-class problems and highlights its advantages in terms of simplicity and performance.