SAMME and Multiclass AdaBoost

Original AdaBoost was built for two-class problems, where a weak learner only needs to beat 50% accuracy to be useful. Extending it naively to $K$ classes breaks down, because a weak learner only needs to beat $1/K$ accuracy (random guessing among $K$ options) to still be informative — with 10 classes, 15% accuracy is actually a strong signal, not a weak one, but classic AdaBoost's weighting formula would treat it as barely better than useless.

Worked example. With 4 classes, random guessing gets 25% accuracy. A weak learner that achieves 35% accuracy is doing meaningfully better than chance, even though it looks weak in absolute terms. SAMME's weight formula, $\alpha = \log\left(\frac{1-\text{err}}{\text{err}}\right) + \log(K-1)$, adds $\log(3) \approx 1.10$ to the standard AdaBoost weight for this 4-class case — ensuring the learner still receives a sizable positive vote instead of being penalized as if it needed to beat 50%. Without that correction term, a formula calibrated for two classes would undervalue every weak learner as the number of classes grows.

A related variant, SAMME.R, uses class-probability estimates from each weak learner (similar in spirit to Real AdaBoost) rather than hard class labels, typically converging faster and to a better solution — most modern boosting libraries implement SAMME.R as their multiclass default.