Near-Zero-Variance Feature Removal

A dataset with hundreds of engineered features almost always contains a handful that are nearly constant — a flag that's 1 for 99.9% of rows, a metric that only ever takes two values with one occurring once. Such a feature carries almost no information for distinguishing observations from one another, yet it still adds noise, slows training, and in small samples can produce unstable coefficient estimates because a model has almost nothing to learn its relationship from.

Near-zero-variance filtering removes these columns before modeling, typically flagging a feature if either of two conditions holds: the ratio of its most common value's frequency to its second most common value's frequency is very high (say, above 95:5), or the number of distinct values is tiny relative to the sample size. Both conditions catch the same underlying problem from different angles — a feature that is effectively constant even if not exactly constant, such as a binary flag that fires on only three trades out of ten thousand.

This is a purely mechanical, model-agnostic cleaning step done early in a pipeline, distinct from more sophisticated feature-selection methods like Lasso regularization that weigh a feature's predictive contribution — near-zero-variance filtering just asks whether a feature varies enough to possibly matter before any model ever sees it, which also prevents rare degenerate columns from silently breaking downstream scaling or encoding steps.