Benchmark Selection for Backtests

A backtest report showing "12% annualized return, Sharpe 1.1" is meaningless on its own — it only becomes useful once you know what it's being compared against. Twelve percent looks great next to a strategy that lost money, and mediocre next to a passive index fund that returned 15% for free. Choosing what to compare a backtest to is not a footnote; it's the step that decides whether the backtest is telling you the strategy adds value or just measuring how much of the market's own return got relabeled as skill.

What makes a benchmark fair

A fair benchmark should share the strategy's investable universe, its market exposure, and roughly its risk level, so that the comparison isolates what the strategy is actually contributing. A long-only US equity stock-picker should be measured against a broad-market index like the S&P 500 or Russell 3000, not against cash — beating cash by 12% a year is trivial in a year the market returned 20%. A market-neutral long-short strategy, by contrast, should be measured against a near-zero exposure benchmark like the risk-free rate, since it isn't supposed to carry market beta at all; comparing it to the S&P 500 would unfairly penalize it in bull years and unfairly flatter it in bear years. A sector-focused strategy should use a sector index, not the broad market, or you risk crediting the strategy for simply being in the right sector.

Worked example: two "successful" backtests, one real edge

Strategy X returns 14% a year with the S&P 500 at 10% over the same backtest period — an excess return of 4 percentage points. Strategy Y returns 9% a year while a market-neutral risk-free benchmark sits at 4% — an excess return of 5 percentage points. On raw returns, X looks better (14% beats 9%). Against the correct benchmark, Y actually shows the larger edge (5 points versus 4). If a researcher compared both strategies to the S&P 500 by mistake, Y would look like it destroyed value (9% versus 10%) when it was actually the stronger result once its market-neutral mandate is accounted for. The benchmark isn't a cosmetic detail here — it flips which strategy looks better.

What this means in practice

Get the benchmark decision wrong and every downstream number — alpha, Sharpe relative to peers, "beat the market" claims — inherits the error. Practitioners typically pick a benchmark before running the backtest, not after seeing the results, precisely to avoid the temptation to shop around for whichever comparison makes the numbers look best. When a strategy's exposures shift over time (say, a long-short book that drifts net long), the benchmark should be revisited rather than left as a historical artifact.