Hard vs Soft Voting
Hard voting combines an ensemble's predictions by majority vote of final class labels, while soft voting averages the underlying predicted probabilities first — soft voting usually wins when those probabilities are meaningful.
An ensemble of classifiers needs a rule for turning several individual predictions into one final answer. Hard voting takes each model's final predicted class and picks whichever class gets the most votes, like a simple majority election. Soft voting instead averages the predicted probabilities each model assigns to each class, and picks the class with the highest average probability — using more information than a bare label.
Hard voting only sees each model's final class choice, treating a 51%-confident vote the same as a 99%-confident one; soft voting averages the actual probabilities, so confident predictions carry more weight than borderline ones — which is why soft voting typically outperforms hard voting whenever the underlying probability estimates are reasonably well-calibrated.
Worked example. Three classifiers vote on whether a loan defaults. Model A says default with 51% confidence, Model B says default with 95% confidence, and Model C says no-default with 60% confidence. Under hard voting, it's 2 votes for default versus 1 for no-default, so the ensemble predicts default. Under soft voting, the average default probability is — still predicting default here, but the margin now reflects how confident the majority actually was, not just a head count. Change Model A's estimate to a barely-over-the-line 51% no-default flip, and hard voting's outcome can swing entirely on a coin-flip-close call, while soft voting's average moves only slightly.
Soft voting requires that each model's probability outputs actually mean something (are reasonably calibrated); if one model's probabilities are systematically overconfident or poorly scaled, it can dominate the average and drag soft voting's accuracy below hard voting's simpler majority rule.
Related concepts
Practice in interviews
Further reading
- Kuncheva, Combining Pattern Classifiers: Methods and Algorithms (ch. 4)