Building a Purged Walk-Forward Harness

Standard k-fold cross-validation splits data into random chunks, trains on some, tests on the rest, and rotates. It works well when rows are independent. Financial data is not independent: a label built from "the stock's return over the next 20 days" overlaps in time with every other label whose 20-day window touches the same dates. If a random split puts day 100's row in the training set and day 105's row in the test set, the model has effectively already seen most of the test label's outcome, because the two labels share 15 of their 20 days. A model evaluated this way looks far better than it will ever perform live, and the gap is not a small correction — it can be the entire reported edge.

Purging and embargo, in words before symbols

A purged walk-forward harness fixes this with two related but distinct mechanisms.

Purging removes any training sample whose label window overlaps the test window. If the test set covers dates 100–120 and a training sample's label was built from returns spanning dates 95–115, that sample overlaps the test period and must be dropped from training — even though its own "date" is 95, safely before the test set starts.

Embargo adds a buffer of extra dates immediately after the test period during which no training samples are allowed, because information can leak in the other direction too: a feature computed on day 121 might use market data that reflects information released during the test window (earnings, a data revision, a cross-sectional ranking that used test-period cross-section for normalisation). The embargo is a fixed number of dates cleared on both sides of every split as a margin of safety.

Walk-forward itself is the structural choice underneath both: instead of one random train/test split, the harness advances through time in order — train on an expanding or rolling window, test on the period immediately after, then roll forward and repeat — so evaluation always mimics the one thing a live strategy actually does: use only the past to predict the future.

A worked example

Suppose the label is a 10-day forward return, and the harness is testing the fold covering days 200–230.

• Any training sample with a label window that touches day 200–230 must be purged. A sample dated day 195 has a label window of 195–205 — it overlaps the test set by 5 days, so it's dropped, even though "day 195" looks safely in the past.
• After the test set ends at day 230, an embargo of, say, 10 days (matching the label horizon) blocks training samples from days 231–240 as well, because a feature on day 231 could already reflect information that only became fully known once day 230's outcomes settled.
• The next training window then legitimately starts at day 241.

Compare the effective sample size lost: on a horizon of 10 days with an embargo of 10 days, roughly 20 days of data around every test fold are unusable for training. On a five-fold walk-forward split of two years of data, this purging can remove a meaningful fraction of the nominal dataset — which is the honest price of a leakage-free estimate, not a bug to work around.

Picture two overlapping windows sampled from this path: a "training" segment and a "test" segment placed close together in time. Where the paths are still correlated — near the boundary — is exactly the region a purged harness discards. The purge width should track how far apart two points on the path need to be before their fluctuations become genuinely independent, which is precisely what the label horizon plus embargo is trying to estimate.

Where this goes wrong in practice

The most common failure isn't skipping purging outright — most desks know to purge — it's purging the label overlap but forgetting that some features are themselves built from rolling windows (a 60-day volatility estimate, a cross-sectional rank computed over the whole current universe) that carry the same overlap problem. A feature computed with a 60-day lookback on day 205 has already absorbed information from days 146–205, which reaches into the test window's neighbourhood in the same way a label would. A harness that only purges on label overlap and ignores feature lookback windows still leaks, just less obviously.