Writing a Strategy Hypothesis Spec

A doctor doesn't order every test in the hospital and then look for something abnormal in the results — they form a specific hypothesis first ("I suspect appendicitis because of the pain location and fever pattern") and order the test that would confirm or rule it out. Ordering everything and mining for an anomaly afterward finds something every time, whether or not it's real. Strategy research has the identical trap: running a backtest engine over thousands of parameter combinations until one looks profitable is the financial equivalent of ordering every test and calling whatever lights up "the diagnosis." A hypothesis spec is the discipline of writing down, before touching the data, exactly what you expect to find and why — in a form specific enough that the data could prove you wrong.

What the spec has to contain

A minimal hypothesis spec answers four questions in writing, before backtesting begins: what is the pattern (a precise, quantifiable claim, not "momentum works"), why should it exist (an economic mechanism — behavioral bias, structural friction, or risk compensation, not just "the data shows it"), where and when should it show up (which universe, which regime, expected magnitude), and what would prove it wrong (a pre-specified out-of-sample test and a threshold for abandoning the idea).

The "why" clause matters because it constrains the "what" clause. If the economic story is "underreaction to earnings news because analysts are slow to update," the hypothesis predicts the effect should be larger in stocks with less analyst coverage and smaller immediately after the earnings date passes into wide public knowledge — testable, falsifiable sub-claims that follow from the mechanism, not free parameters to be tuned after seeing results.

Worked example 1. A researcher hypothesizes that stocks added to a widely-tracked index earn abnormal returns in the days before the effective addition date, because passive index funds must buy to replicate — a mechanical, liquidity-driven story, not a behavioral one. The spec states expected magnitude in advance: given that passively-tracked assets benchmarked to the index total roughly $5 trillion and a typical addition represents about 0.02% of index weight, expected forced buying is around $5 \times 10^{12} \times 0.0002$, i.e. $1 billion, concentrated in a stock with, say, $50M average daily volume — over 20 days of that stock's normal trading volume compressed into a short pre-effective window, predicting a measurable price impact. The pre-specified test: measure abnormal return in the 5 days before effective date across all additions from 2015–2023, expecting a positive, statistically significant abnormal return of a magnitude consistent with a price-impact model, not an arbitrary "positive and significant" bar that could be satisfied by almost any effect size.

The counter-example: a spec that fails its own test

Worked example 2. Suppose a second researcher's spec predicts the index-addition effect should also hold in reverse for index deletions — expecting a negative abnormal return before the effective removal date, with a magnitude roughly proportional to the deleted stock's index weight, by the same mechanical mechanism. Running the pre-specified test on 2015–2023 deletions gives an average abnormal return of $-0.3\%$ over the five days before effective date, against a pre-specified threshold of at least $-1.5\%$ (derived the same way as example 1's $1B calculation, scaled for typical deletion weights). The measured effect misses the threshold by a wide margin. Because the spec was written down first, this is a clean, honest null result — the mechanism doesn't hold up as strongly on the deletion side, which is itself useful information (perhaps deletions are pre-announced further in advance, giving funds more time to trade gradually) rather than something to explain away by loosening the threshold after the fact.

What this means in practice

Serious research shops require a written spec, reviewed by someone other than the researcher, before a strategy is allowed to consume backtest compute or move to paper trading — not as bureaucracy, but because the single largest source of false discoveries in quant research is unconstrained, iterative parameter search dressed up after the fact as a coherent thesis (see p-Hacking and the Garden of Forking Paths). The spec becomes the reference document the first live week is checked against (see What to Watch in a Strategy's First Week).