Combinatorial Purged Cross-Validation

A single backtest gives you a single equity curve, which is a single draw from a distribution you never get to see. Maybe the strategy is good; maybe that one path just happened to dodge the bad patches. Ordinary Walk-Forward Analysis testing suffers this badly: it produces exactly one out-of-sample path, so you can't tell skill from luck. Combinatorial Purged Cross-Validation (CPCV), from López de Prado, squeezes many out-of-sample paths out of the same data, turning your one equity curve into a whole distribution you can actually reason about.

The idea builds directly on Purged & Embargoed Cross-Validation: it keeps the purging (drop training labels whose forward windows overlap the test block) and the embargo (drop a buffer after it), then adds combinatorics on top.

How it works

Split the time series into $N$ equal, ordered groups. Instead of testing on one group at a time, test on every combination of $k$ groups at once, purging and embargoing each. There are $\binom{N}{k}$ such combinations. Because each combination leaves a different set of groups for testing, you can stitch the test predictions together into complete, start-to-finish backtest paths, and it turns out you get many of them. The number of distinct paths is

$$\varphi[N, k] = \frac{k}{N}\binom{N}{k}.$$

Here $N$ is how many groups you carve the timeline into, $k$ is how many you hold out for testing in each combination, and $\binom{N}{k}$ is "$N$ choose $k$", the count of ways to pick those $k$ groups. Each of the $\varphi$ paths is a full equity curve built from models that never saw their own test data (thanks to purging and embargo), so you end up with $\varphi$ Sharpe ratios instead of one.

The table shows how quickly paths accumulate. Testing one group at a time ($k=1$) recovers plain k-fold and gives just one path; holding out two or more groups unlocks many.

Groups $N$	Held out $k$	Combinations $\binom{N}{k}$	Backtest paths $\varphi$
6	1	6	1
6	2	15	5
10	2	45	9
10	5	252	126

A modest number of groups yields a rich set of backtest paths. With 6 groups holding out 2, the same data produces 15 train/test splits that reassemble into 5 full-length out-of-sample paths.

Worked example

Take $N = 6$ groups and hold out $k = 2$ at a time. There are $\binom{6}{2} = 15$ ways to choose the test pair, so you train and evaluate $15$ models (each with the overlapping labels purged and a buffer embargoed). Those 15 splits recombine into $\varphi = \tfrac{2}{6}\times 15 = 5$ complete backtest paths.

Now instead of one Sharpe you have five, say $\{0.9,\ 1.3,\ 0.4,\ 1.1,\ 0.7\}$. Immediately you know things a single number could never tell you: the mean is about $0.88$, but the spread runs from $0.4$ to $1.3$. If your reported single-path Sharpe had been the $1.3$, you'd now see it was the lucky top of the range, not typical. You can also count how often the strategy's in-sample best configuration underperforms out-of-sample across these paths, which is precisely the ingredient for estimating the probability of Backtest Overfitting.

Pitfalls

• Paths are not independent. The $\varphi$ paths share data and overlapping models, so their Sharpes are correlated. Treat the spread as an honest picture of variability, not as $\varphi$ independent samples for a tight confidence interval.
• Purge and embargo still mandatory. CPCV without purging leaks exactly like naive k-fold; the combinatorics multiply the leakage rather than fixing it. See Purged & Embargoed Cross-Validation.
• Cost. You train $\binom{N}{k}$ models, which explodes for large $N$ and mid-range $k$. Choose $N$ and $k$ for enough paths (a few dozen) without a runaway model count.
• Group size vs label span. Groups must be long relative to the label horizon from Triple-Barrier Labeling, or purging eats most of each fold.

In interviews

If asked "how do you know your backtest's Sharpe wasn't a fluke of one particular data path?", the strong answer is CPCV: partition into $N$ groups, test on all $\binom{N}{k}$ combinations with purging and embargo, and reassemble $\varphi = \frac{k}{N}\binom{N}{k}$ out-of-sample paths to get a distribution of Sharpes. Note that this distribution is what feeds the probability-of-backtest-overfitting estimate, and that the paths are correlated so you shouldn't oversell their precision.