Quant Memo
Advanced

Purged & Embargoed Cross-Validation

López de Prado's fix for validating models on labeled financial data, purge training observations whose label windows overlap the test set, then embargo a buffer after it to kill serial-correlation leakage.

Prerequisites: Cross-Validation for Time Series, Triple-Barrier Labeling

Purged and embargoed cross-validation is López de Prado's remedy for the leakage that makes ordinary Cross-Validation for Time Series invalid on labeled financial data. The core problem is that financial labels are built from forward-looking windows, so an observation in the training set can share outcome information with an observation in the test set even when the two are separated in time. Purging removes the overlapping training observations; embargoing removes a further buffer to neutralize serial correlation. Together they restore the informational separation that cross-validation assumes.

The overlap problem, precisely

Assign each observation ii a label span [ti,0,ti,1][t_{i,0}, t_{i,1}]: the interval of time whose price path determines its label. With Triple-Barrier Labeling, ti,0t_{i,0} is the event time and ti,1t_{i,1} is the moment a barrier is touched (at most ti,0+ht_{i,0}+h). Two observations ii and jj are informationally dependent whenever their spans intersect:

[ti,0,ti,1][tj,0,tj,1].[t_{i,0}, t_{i,1}] \cap [t_{j,0}, t_{j,1}] \neq \varnothing.

Standard CV ignores this and freely places overlapping observations on opposite sides of the split, leaking the test labels into training. Because the labels are correlated, the model is graded partly on data it effectively trained on, and the cross-validated score is biased upward.

Purging

Purge from the training set every observation whose label span overlaps the label span of any test observation. If the test set covers label spans over [t,t][\underline{t}, \overline{t}], drop training observation ii whenever

ti,1tandti,0t.t_{i,1} \ge \underline{t} \quad\text{and}\quad t_{i,0} \le \overline{t}.

This removes both the training points whose forward windows reach into the test period and those whose windows are reached into by it. After purging, no surviving training label shares information with any test label, so the fundamental leakage channel is closed.

Embargo

Purging handles direct label overlap, but serial correlation in features and returns means observations just after the test period still carry information about it, momentum, volatility persistence, slow information diffusion. To handle this, embargo a buffer: drop training observations that fall within a small window after the end of the test set. The embargo length is a fraction δ\delta of the total sample,

hembargo=δT,h_{\text{embargo}} = \lceil \delta\, T \rceil,

with δ\delta on the order of the label horizon relative to the sample (often ~1%). Any training observation whose event time lies in (t,t+hembargo](\overline{t}, \overline{t} + h_{\text{embargo}}] is removed. The embargo is one-sided, applied after the test block, because that is the direction in which residual autocorrelation leaks once purging has removed the pre-test overlap.

Combinatorial purged CV

Because a single train/test split gives one noisy path, López de Prado generalizes to Combinatorial Purged Cross-Validation (CPCV): partition the series into NN ordered groups, choose all (Nk)\binom{N}{k} ways to assign kk groups to testing, and purge+embargo each combination. This yields many backtest paths from the same data, turning a single equity curve into a distribution, and feeds directly into estimating the Backtest Overfitting probability via combinatorially-symmetric CV.

Worked example

Daily observations, labels = the triple-barrier outcome over at most h=5h=5 trading days, sample length T=2,000T = 2{,}000 days, embargo fraction δ=1%\delta = 1\% so hembargo=20h_{\text{embargo}} = 20 days. A test fold spans days 1,0001{,}0001,2001{,}200.

  1. Purge: any training observation with an event time in, say, days 996996999999 has a 5-day forward window reaching into day 1,0001{,}000+, so its label overlaps the test block, drop it. Likewise drop training points whose windows are entered by the test set. This removes a thin band of ~5 days on each side of the test block.
  2. Embargo: additionally drop training observations in days 1,2011{,}2011,2201{,}220 (the hembargo=20h_{\text{embargo}}=20-day buffer after the test block), because volatility and momentum from the test period persist into them.

What remains as training is genuinely separated from the test labels in information, not merely in index. Skipping either step lets the model post a cross-validated accuracy several points higher than it deserves, the exact illusion that sinks live-traded ML strategies.

Failure modes

  • Purging without embargo: direct overlap is removed but serial-correlation leakage in the post-test window remains, still inflating scores.
  • Embargo too small relative to the autocorrelation length of features.
  • Wrong label spans: if ti,1t_{i,1} is mis-recorded (e.g. using a fixed horizon when a barrier was touched earlier, or vice versa), the purge removes the wrong observations.
  • Full-sample preprocessing applied before splitting, scalers and PCA must be fit inside each purged training fold, or Look-Ahead Bias re-enters through the back door.
  • Treating CPCV paths as independent when estimating variance; they share data and are correlated.

In interviews

The setup: "You label observations with 10-day forward returns and want to cross-validate a classifier, how do you avoid leakage?" A complete answer defines the label span, describes purging (remove training observations whose forward windows overlap the test block) and embargo (drop a post-test buffer to kill serial-correlation leakage), and notes that transforms must be fit inside the fold. A strong candidate adds that CPCV produces many purged paths, which is what you need to estimate the probability of backtest overfitting. The one-line principle: cross-validation is valid only when train and test are independent in information, on labeled financial data you must purge overlapping labels and embargo the autocorrelated aftermath to make that true.

Related concepts

Practice in interviews

Further reading

  • López de Prado, Advances in Financial Machine Learning (Ch. 7)
  • López de Prado, The 10 Reasons Most Machine Learning Funds Fail
  • Bergmeir & Benítez, On the Use of Cross-Validation for Time Series Predictor Evaluation
ShareTwitterLinkedIn