Adversarial Backtest Design

A researcher who builds a strategy is the worst possible person to try to break it — not through dishonesty, but because every choice they've already made (the universe, the lookback, the entry filter) survived precisely because it looked good on the data they were staring at. Asking that same researcher "does this survive stress testing?" is asking someone to find flaws in their own reasoning, which is a request people are reliably bad at fulfilling. Adversarial backtest design puts a second person, with no stake in the strategy's success, explicitly in charge of trying to kill it.

What the adversary is asked to do

The adversarial reviewer's job is not to check the code runs. It's to actively hunt for the specific failure modes that make a backtest lie: shift the start and end dates and see if the Sharpe survives; drop the three best trades and see what's left; swap in a slightly different but equally defensible universe definition (add the delisted names, change the market-cap cutoff) and see if the edge holds; widen transaction cost assumptions until they hurt; check whether performance is concentrated in one regime, one sector, or a handful of trades. A strategy that only works under the exact conditions the original researcher happened to choose was never robust — it was tuned to those conditions, whether anyone intended that or not.

Worked example

A pairs-trading strategy in energy futures reports Sharpe 1.9 on its original backtest, 2015–2022, 45 pairs, standard 20-day z-score entry. An adversarial reviewer runs four stress checks:

Stress test	Result
Original	Sharpe 1.9
Shift start date by 6 months	Sharpe 1.1
Remove best 5 of 340 trades	Sharpe 0.6
Widen bid-ask assumption by 50%	Sharpe 0.7
Extend universe to 60 pairs (equally defensible cutoff)	Sharpe 0.9

Every stress test degrades the number substantially, and removing five trades out of 340 — under 1.5% of the sample — cuts the Sharpe by more than half. That concentration is the tell: the strategy's headline number depended on a handful of outlier trades and a specific date window, not on a repeatable edge. The adversarial pass caught in an afternoon what the original backtest's authors, having built and lived inside the strategy for months, never went looking for.

Adversarial review works best paired with a written validation protocol, so the adversary knows the pass bar before running any stress test — otherwise the debate about whether a given degradation is "acceptable" happens after the fact, which reopens exactly the hindsight problem the review was meant to close. See Independent Replication by a Second Researcher for the complementary check of rebuilding the strategy from scratch rather than stress-testing the existing implementation, and Monte Carlo Permutation Tests for Strategies for a systematic, automated version of the same adversarial instinct.