Informative Missingness and Indicator Features

It's tempting to treat missing data as a nuisance to be patched over — fill it with the column mean, move on. But sometimes a value is missing for a reason that matters: an earnings estimate is absent because no analyst wanted to touch a name mid-scandal, a quote is missing because the venue halted trading, a loan's collateral value is blank because the borrower stopped reporting it. In each case, the missingness itself carries information about the outcome you're trying to predict — arguably more than whatever value would have filled the gap.

The idea: missingness is not always noise, so encode it explicitly

There's a standard three-way taxonomy for why data is missing: missing completely at random (no relationship to anything), missing at random (related to other observed variables but not to the hidden value itself), and missing not at random — where the fact of being missing is directly related to the unobserved value or the outcome. It's this last case that matters most in finance, because naively imputing a value (say, filling with the column mean) throws away the informative part and can actively mislead the model.

The fix has two parts, both applied together: first, create a binary indicator feature,

$$m_i = \mathbb{1}[x_i \text{ is missing}]$$

second, impute the missing value with some reasonable placeholder (mean, median, or a model-based estimate) so the model still has a number to work with in that column. In plain English: $m_i$ equals 1 for a row where $x_i$ was missing and 0 otherwise — it lets the model learn a separate effect for "this value was missing" on top of whatever effect the (now imputed) underlying value has, instead of forcing the missingness signal and the imputed placeholder to fight for the same coefficient.

Worked example: analyst coverage as a missingness signal

Suppose "consensus earnings estimate" is missing for 40 out of 500 stocks in a dataset, and those 40 stocks subsequently underperform the market by an average of 6% over the next quarter — because a lack of analyst coverage is itself correlated with distress, low liquidity, or accounting uncertainty that analysts are avoiding. If the missing values are simply mean-imputed with no indicator, the model sees 40 stocks with an unremarkable, average-looking earnings estimate and no way to flag that those 40 rows are actually different in kind. Adding the indicator estimate_missing = 1 for those 40 rows lets the model learn directly that "no coverage" predicts underperformance — a real, usable signal that mean-imputation alone would have erased.

What this means in practice

Always pair imputation with a missingness indicator when there's any reason to suspect the missingness pattern isn't purely random — which, in financial data, is closer to the norm than the exception, since data providers, analysts, and reporting requirements all respond to the same distress signals a model is trying to detect. The indicator costs almost nothing to add and, when missingness turns out to be uninformative after all, the model simply learns to ignore it; the risk is entirely asymmetric in favor of including it.