Pre-Registration of Research Hypotheses

A researcher testing a momentum signal on ten years of stock data has enormous freedom while building the test — which lookback window to use, which universe to trade, how to handle rebalancing frequency, which risk adjustment to apply. If the results are only checked after all these choices are locked in, that's a clean test. But in practice it's tempting, and often unconscious, to try a few reasonable variations, see which one produces the best-looking backtest, and present that version as "the" result — quietly turning a single hypothesis test into many, without adjusting for having effectively searched over choices until something worked.

Committing before looking

Pre-registration means writing down, before running the test, exactly what will be tested and how: the hypothesis, the exact parameters, the universe, the sample period, and — crucially — the criteria for what counts as success. That document is timestamped and kept unchanged. When the actual results come in, they're compared against what was pre-committed, not against whatever version of the analysis happens to look best. This doesn't forbid trying variations afterward — a researcher can absolutely explore further once the pre-registered test is run — but it draws a clear line between "this is the confirmatory test I committed to" and "this is exploratory follow-up," and only the former should be treated as strong evidence.

A concrete case

A researcher believes a certain earnings-surprise signal predicts next-month returns. Before running anything, they write down: signal defined as reported EPS minus consensus estimate, tested on US large caps from 2010–2023, holding period one month, success defined as a statistically significant positive return with a t-statistic above 2. They run exactly that test. If it works, the pre-registration record shows this wasn't the fifth variant that happened to clear the bar — it was the one specified in advance. If it doesn't work, they're free to explore why and try adjustments, but any resulting strategy from that further exploration is now an exploratory finding, which reasonably needs a separate, later confirmatory test of its own before being trusted the same way.

What this means in practice

Pre-registration is most valuable for the results that matter most — the ones that will drive a capital allocation decision — where the temptation to quietly favor a flattering variant is strongest. It costs almost nothing (a short document, written a few minutes before running the analysis) and directly targets one of the most common and hardest-to-detect ways a backtest result ends up looking better than the real, out-of-sample strategy will perform.