Hansen's Superior Predictive Ability Test
A statistical test that checks whether the single best strategy found among many candidates is genuinely better than a benchmark, after properly accounting for the fact that testing many strategies makes some look good by luck alone.
If a researcher backtests a thousand trading rules against a benchmark, some of them will beat it by chance alone, even if none of them has any genuine edge — the more candidates tried, the more likely the best-looking one is just a lucky draw. Comparing only the single best strategy's performance to the benchmark using an ordinary significance test badly overstates how impressive that result really is, because it ignores how many other candidates were tried and discarded along the way.
Hansen's superior predictive ability (SPA) test corrects for this by asking a sharper question: given all the candidate strategies that were actually tested, what is the chance that the best-performing one would look this good purely from luck, if none of the candidates truly had any edge over the benchmark? It builds this null distribution using a bootstrap that resamples the historical return data many times, tracking how good the best of many equally-skill-less strategies would appear to be just from sampling noise, and compares the real best strategy's performance against that distribution rather than against a naive single-strategy benchmark.
The SPA test answers "is the winner of a strategy search actually good," not "is this one strategy good" — by simulating how good the best of many worthless strategies would look purely by chance, and checking whether the real winner clears that higher, luck-adjusted bar.
It is considered an improvement on the earlier White's Reality Check because it corrects a known conservatism in that test, giving a fairer (though still appropriately strict) verdict when hundreds of variations of essentially the same idea have been tried.
Related concepts
Further reading
- Hansen, 'A Test for Superior Predictive Ability'