Confirmation Bias at the Research Desk
The tendency to notice, trust, and dig deeper into evidence that supports an idea you already like, while glossing over evidence against it — and the concrete habits that catch it in day-to-day research work.
Prerequisites: Defining Success Criteria Before You Look
A researcher who's spent two weeks on a signal, and is genuinely excited about it, looks at a mixed result differently than a skeptical outsider would. A positive sub-period gets read as "the signal working as expected." A negative sub-period gets read as "probably a regime where this factor is known to struggle" — an explanation reached for automatically, without applying the same scrutiny that a positive result got waved through with. Neither reaction is dishonest; it's confirmation bias, the very ordinary human tendency to accept confirming evidence at face value while demanding extra justification before accepting disconfirming evidence, and it shows up especially strongly once someone has invested real time in an idea they'd like to be right.
Where it sneaks in during research
It shows up in which sub-periods or sub-universes get scrutinized: a researcher checking "does this work in small caps too" only after the large-cap result already looked good, and stopping the check the moment a supportive-enough number appears rather than continuing to look. It shows up in how much effort goes into explaining away a bad result versus a good one — a bug hunt that's thorough when the result disappoints and cursory when the result pleases. And it shows up subtly in which papers or prior results get cited as support: the ones that agree get remembered and reached for, the ones that don't get quietly forgotten.
Worked example
A researcher tests a momentum-based signal and gets a strong result in US equities. Curious whether it generalizes, they check European equities and get a weak, borderline result. Confirmation bias nudges toward "Europe probably has structural differences that dampen momentum, consistent with known literature" — true in general, but reached for immediately here specifically because it lets the US result stand unchallenged. A more even-handed read would ask the same question of both results: is the European result weak because of real structural differences, or because the US result was itself partly an artefact of that particular period, and the European data is revealing that more honestly? Both explanations deserve equal digging, not just the one that protects the original finding.
What this means in practice
The defense against confirmation bias isn't willpower — nobody reliably catches their own bias by trying harder to be objective in the moment. It's structural habits set up in advance: pre-registered success criteria, a colleague specifically asked to argue the other side, or testing on held-out data the researcher genuinely hasn't seen yet. These work precisely because they don't depend on noticing your own bias as it happens, which is the one thing confirmation bias reliably prevents.
Confirmation bias means confirming evidence gets accepted easily while disconfirming evidence gets explained away — a bias that grows stronger the more invested someone already is in an idea. Structural defenses set up before results are known work better than trying to catch the bias in the moment.
Further reading
- Kahneman, Thinking, Fast and Slow