Null Results and Why They Are Worth Keeping
A backtest or signal that didn't work is still evidence, recording it prevents the whole team from re-testing the same dead idea six months later and helps calibrate how much to trust a positive result.
A "null result" is a signal, factor, or model variant that was tested carefully and simply didn't show a meaningful edge. The instinct on most desks is to throw these away, nobody writes a research note about the thing that didn't work, but that instinct is expensive at team scale. If idea A was tested and rejected in January and nobody wrote it down, a new analyst in June will spend a week rediscovering that idea A doesn't work, because there was no record to stop them.
Keeping null results also matters statistically, not just operationally. If a team tests one hundred candidate signals and only reports the five that looked good, an outside reader (or the team itself, a year later) has no way to judge whether those five are real edges or the expected number of false positives you'd get by chance from testing a hundred things. A log of what was tried and rejected turns an unknown multiple-testing problem into a known one.
Concrete illustration. A research log entry for a null result should record: the exact hypothesis, the data and time window used, the test performed, the result (with confidence interval, not just "didn't work"), and the researcher's best guess at why. "Tested 12-month price momentum on small-cap ADRs, 2015–2023, information ratio 0.08 (not significant at any reasonable threshold), likely because ADR liquidity is too thin for the signal to be tradeable" is reusable a year later in a way that a missing entry never is.
A null result recorded with the same rigor as a positive one, hypothesis, data, method, outcome, prevents duplicated work and lets anyone assess how many ideas were tried before a "significant" one turned up, which is essential context for judging whether that significant result is real.
Discussion
💡 Discussion rules
- Ask and answer about this concept. Off-topic gets removed.
- No homework dumps. Show what you tried first.
- Corrections are welcome. Cite a source when you claim an error.
Loading discussion…
Related concepts
Further reading
- Practitioner convention on research logs and idea graveyards