Bernoulli, Multinomial and Complement Naive Bayes

Naive Bayes classifiers all share the same core idea — estimate the probability of a class from each feature independently and multiply the results together — but they differ in what kind of feature data they assume, which matters a lot for something like classifying documents by their words. Think of it like three different ways of describing a shopping basket: did you buy milk or not (yes/no), how many cartons of milk did you buy (a count), or how unusual is your basket relative to what people in other stores typically buy.

Bernoulli naive Bayes treats each feature as present or absent, ignoring how many times a word appears in a document — useful when the mere presence of a word (like a rare technical term) is what matters, not its frequency. Multinomial naive Bayes instead counts occurrences, so a document mentioning "earnings" five times contributes more evidence toward a "financial news" class than one mentioning it once, which fits most text classification tasks better since word frequency does carry information. Complement naive Bayes is a variant of the multinomial version built specifically for imbalanced classes: instead of estimating word statistics from the class itself, it estimates them from everything not in that class, which turns out to give more stable estimates when one class has far fewer training examples than the others.

In practice, multinomial is the default starting point for text classification, Bernoulli is preferred for short or sparse text where presence matters more than count, and complement is reached for once class imbalance is degrading the multinomial version's performance.