Causal Features vs Merely Predictive Features

A predictive model doesn't distinguish between two very different ways a feature can be useful. It can be useful because it captures a genuine mechanism — some real, structural reason it moves ahead of or alongside the outcome. Or it can be useful because, during the specific window of data the model was trained on, it happened to correlate with the outcome through some third factor, a shared trend, or plain historical accident. Standard measures of usefulness — correlation, feature importance, out-of-sample R² on a held-out slice of the same period — score both kinds of feature identically, because both reduce prediction error on data drawn from the same regime that produced the correlation in the first place.

The distinction matters because the two kinds of feature respond completely differently to a regime change. A causal feature keeps predicting because the mechanism behind it is still operating — earnings surprises still move prices because information still gets priced in, no matter what year it is. A merely predictive feature can flip sign, vanish, or become actively harmful the moment the coincidental conditions that produced the correlation change, because there was never a mechanism there to persist. This is exactly the distinction invariant risk minimization tries to detect automatically by checking stability across environments — but the underlying idea is worth understanding on its own: does this feature predict returns because of something, or merely alongside something that happened to also move returns during the sample you have?

Worked example

A quant finds that the daily count of news articles mentioning a stock strongly predicts next-day returns in a 2016–2019 backtest, with feature importance rivaling standard momentum and value factors. Two candidate explanations exist. Causal story: more news genuinely means more new information reaching the market, and prices adjust to it with a short lag — a real, structural information-diffusion mechanism that should persist across periods. Coincidental story: during 2016–2019, retail order flow specifically chased whatever stocks were in the news, a pattern of investor behavior tied to that era's brokerage app design and social-media habits, not to information content itself. When retail-brokerage design changed materially in 2020–2021, the feature's predictive power collapsed and its sign became unstable — evidence retroactively favoring the coincidental story. A momentum factor built on price action itself, tested over the same two windows, kept a stable (if smaller) relationship to future returns in both, consistent with reflecting a more durable mechanism.

predictive-only feature news volume next-day return retail flow, era-specific breaks when era changes

What this means in practice

The cheapest diagnostic available is testing a feature's relationship to the outcome across genuinely distinct sub-periods, market regimes, or asset classes rather than only checking in-sample or single-period out-of-sample fit — a feature whose relationship holds up across distinct regimes is a much stronger candidate for being causal, and thus durable, than one that only worked in the exact window it was discovered in. This doesn't replace an actual argument for why a feature should be causal, but instability across regimes is close to a decisive vote against relying on it going forward.