Benchmark Selection for Backtests

A backtest report showing "12% annualized return, Sharpe 1.1" is meaningless on its own, it only becomes useful once you know what it's being compared against. Twelve percent looks great next to a strategy that lost money, and mediocre next to a passive index fund that returned 15% for free. Choosing what to compare a backtest to is not a footnote; it's the step that decides whether the backtest is telling you the strategy adds value or just measuring how much of the market's own return got relabeled as skill.

What makes a benchmark fair

A fair benchmark should share the strategy's investable universe, its market exposure, and roughly its risk level, so that the comparison isolates what the strategy is actually contributing. A long-only US equity stock-picker should be measured against a broad-market index like the S&P 500 or Russell 3000, not against cash, beating cash by 12% a year is trivial in a year the market returned 20%. A market-neutral long-short strategy, by contrast, should be measured against a near-zero exposure benchmark like the risk-free rate, since it isn't supposed to carry market beta at all; comparing it to the S&P 500 would unfairly penalize it in bull years and unfairly flatter it in bear years. A sector-focused strategy should use a sector index, not the broad market, or you risk crediting the strategy for simply being in the right sector.

Worked example: two "successful" backtests, one real edge

Strategy X returns 14% a year with the S&P 500 at 10% over the same backtest period, an excess return of 4 percentage points. Strategy Y returns 9% a year while a market-neutral risk-free benchmark sits at 4%, an excess return of 5 percentage points. On raw returns, X looks better (14% beats 9%). Against the correct benchmark, Y actually shows the larger edge (5 points versus 4). If a researcher compared both strategies to the S&P 500 by mistake, Y would look like it destroyed value (9% versus 10%) when it was actually the stronger result once its market-neutral mandate is accounted for. The benchmark isn't a cosmetic detail here, it flips which strategy looks better.

What this means in practice

Get the benchmark decision wrong and every downstream number, alpha, Sharpe relative to peers, "beat the market" claims, inherits the error. Practitioners typically pick a benchmark before running the backtest, not after seeing the results, precisely to avoid the temptation to shop around for whichever comparison makes the numbers look best. When a strategy's exposures shift over time (say, a long-short book that drifts net long), the benchmark should be revisited rather than left as a historical artifact.