Laplace Smoothing for Naive Bayes

A Naive Bayes classifier estimates the probability of each feature value given each class straight from training-data frequencies — for instance, "of all spam emails in training, what fraction contained the word 'refinance'." The trouble is multiplicative: Naive Bayes combines these per-feature probabilities by multiplying them together, and if even one feature value never appeared in training for a given class, its estimated probability is exactly zero, which zeroes out the entire product regardless of how strongly every other feature points the other way.

Laplace smoothing (also called additive smoothing) fixes this by pretending you've seen every possible feature value one extra time (or some other small constant, $\alpha$) beyond what training data actually showed, before computing frequencies. Concretely, instead of estimating $P(\text{word} \mid \text{class}) = \text{count} / \text{total}$, you compute $(\text{count} + \alpha) / (\text{total} + \alpha \times \text{number of possible values})$. This guarantees every probability estimate is strictly positive — a brand-new word never seen in spam training gets a small but nonzero probability instead of an instant, unwarranted veto over the whole prediction.

The smoothing constant $\alpha$ is itself a tuning choice: $\alpha = 1$ (plain Laplace smoothing) is the common default, but larger $\alpha$ pulls every estimate further toward a uniform distribution, which helps more when training data is sparse and hurts more when it's abundant enough that the raw frequencies are already reliable.