Surrogate Data Testing

Suppose a backtest finds that a momentum signal produces a Sharpe ratio of 1.2. Is that because the signal captures something real about how prices move, or because prices have properties — volatility clustering, fat tails, mild autocorrelation — that would let almost any similarly-shaped signal look decent purely by chance? Surrogate data testing answers this by building fake price series that preserve the boring statistical properties of the real data but destroy whatever specific structure the strategy claims to exploit, then checking whether the strategy still "works" on the fakes.

The idea

A surrogate series is generated to match the real data on properties that aren't in question — for instance, the same distribution of daily returns, the same volatility, the same autocorrelation of squared returns (volatility clustering) — while scrambling or randomizing the specific feature the strategy is supposed to depend on, such as the sign and ordering of returns over time. A common construction shuffles the sequence of returns (or their phase, in frequency-domain versions) while leaving the marginal distribution and volatility clustering intact.

You then run the exact same strategy logic on many independently generated surrogates and record the distribution of backtested Sharpe ratios that result. If the real data's Sharpe ratio sits comfortably inside that distribution — say, the surrogates produce Sharpe ratios ranging from −0.8 to 1.5 purely from artifacts of volatility clustering and fat tails — the "edge" you found on real data is not distinguishable from what a strategy would produce on data with no genuine temporal structure at all. If the real result sits far out in the tail of the surrogate distribution, that's evidence the strategy is picking up something surrogates don't have.

Worked example

A researcher backtests a signal that buys after two consecutive down days, on the theory of short-term mean reversion, and gets a Sharpe ratio of 0.9 on five years of daily data. To check whether volatility clustering alone could produce this, they generate 500 surrogate series that reshuffle the order of the real daily returns (destroying any day-to-day dependence) while keeping the same set of returns and hence the same volatility and fat-tailed distribution. Running the identical two-down-days rule on all 500 surrogates produces Sharpe ratios ranging from −0.4 to 0.6, with a mean near 0.1. The real result of 0.9 falls well above every surrogate outcome — only 3 of 500 surrogates beat 0.6 — giving a rough p-value under 1%. That's evidence the mean-reversion pattern is a real feature of day-to-day ordering, not an artifact of the return distribution's shape.

What this means in practice

Surrogate data testing is especially valuable for strategies whose edge is a subtle statistical pattern — mean reversion, momentum, volatility timing — rather than an economically obvious relationship, precisely because those are the patterns most likely to be conjured by the raw statistical texture of price data rather than genuine structure. It's cheap to run and doesn't require any out-of-sample data you'd otherwise be saving for a lockbox test, which makes it a good early screen before committing scarce holdout data to a strategy.