Null Results and Why They Are Worth Keeping
A backtest or signal that didn't work is still evidence — recording it prevents the whole team from re-testing the same dead idea six months later and helps calibrate how much to trust a positive result.
A "null result" is a signal, factor, or model variant that was tested carefully and simply didn't show a meaningful edge. The instinct on most desks is to throw these away — nobody writes a research note about the thing that didn't work — but that instinct is expensive at team scale. If idea A was tested and rejected in January and nobody wrote it down, a new analyst in June will spend a week rediscovering that idea A doesn't work, because there was no record to stop them.
Keeping null results also matters statistically, not just operationally. If a team tests one hundred candidate signals and only reports the five that looked good, an outside reader (or the team itself, a year later) has no way to judge whether those five are real edges or the expected number of false positives you'd get by chance from testing a hundred things. A log of what was tried and rejected turns an unknown multiple-testing problem into a known one.
Concrete illustration. A research log entry for a null result should record: the exact hypothesis, the data and time window used, the test performed, the result (with confidence interval, not just "didn't work"), and the researcher's best guess at why. "Tested 12-month price momentum on small-cap ADRs, 2015–2023, information ratio 0.08 (not significant at any reasonable threshold), likely because ADR liquidity is too thin for the signal to be tradeable" is reusable a year later in a way that a missing entry never is.
A null result recorded with the same rigor as a positive one — hypothesis, data, method, outcome — prevents duplicated work and lets anyone assess how many ideas were tried before a "significant" one turned up, which is essential context for judging whether that significant result is real.
Further reading
- Practitioner convention on research logs and idea graveyards