Hindsight Bias and Narrative Fitting

A backtest turns up a surprising result: a factor built from satellite images of parking lots predicts retailer earnings surprises. Once the number is in hand, a plausible story arrives almost instantly — of course parking lot traffic reflects foot traffic, of course foot traffic predicts sales, of course sales predict earnings surprises. The story feels so natural that it's easy to forget nobody predicted this specific result before seeing it; the explanation was built backward, from the answer to a plausible-sounding path that leads to it. That's hindsight bias: a result feels more obvious and more inevitable after the fact than it would have seemed in advance, and the ease of constructing a story for it gets mistaken for evidence that the result is real.

Why the fitted story is dangerous specifically because it's persuasive

The trouble isn't that these after-the-fact stories are always wrong — sometimes the intuitive explanation is exactly right. The trouble is that a fluent, plausible-sounding narrative can be constructed for almost any result, including pure noise, because human pattern-finding is very good at connecting dots after they're already placed. A backtest result born from a spurious correlation in a specific sample period will still get a story that sounds just as reasonable as a story for a genuinely real effect — the fluency of the explanation carries no information about whether the underlying pattern will hold up out of sample.

Worked example

A researcher backtests dozens of candidate signals and one of them — a factor based on how often a company's name appears in a specific type of local news article — shows a striking result over the test period. A narrative arrives easily: local news coverage picks up regional economic activity ahead of official statistics, giving an information edge. It's a good story. But the same researcher tried dozens of other candidate signals that didn't work, and for any one that happened to show a strong result by chance, an equally fluent story could probably be constructed after the fact. The test of whether the story is worth trusting isn't how plausible it sounds — it's whether the effect holds up on a fresh, previously unseen sample the story wasn't built to fit.

What this means in practice

A satisfying explanation for a backtest result is not, by itself, evidence the result is real — it's evidence that humans are good at generating explanations. The genuine safeguard is testing the finding on data that wasn't used to generate the story in the first place; if the pattern only shows up in the sample the narrative was reverse-engineered from, the story explains nothing, no matter how convincing it sounds in a research meeting.