Learning Track · Advanced
← All tracksBacktesting & ML in Finance
How backtests lie, and the validation discipline that separates signal from noise.
Most backtests are wrong, and the ways they're wrong are systematic. This track covers the biases (look-ahead, survivorship, data-snooping), the validation methods that survive time-series dependence (purged and embargoed cross-validation), and how to quantify overfitting (the deflated Sharpe ratio). It then covers machine learning applied to finance honestly, including why naive ML fails here.
It assumes the statistics track and pairs naturally with systematic strategies.
14 of 14 lessons published · progress saves in your browser
- 1Backtest Design
What separates a historical simulation you can trust from a curve-fit, point-in-time data, realistic costs and slippage, and the crucial distinction between a backtest and a forecast.
- 2Look-Ahead Bias
The most common way backtests lie, leaking information into the simulation that was not knowable at decision time, through restated fundamentals, same-bar execution, or full-sample preprocessing.
- 3Survivorship Bias
The upward distortion in backtests that use only the securities or funds that survived to today, quantifying the bias, why delisted names matter, and how point-in-time universes fix it.
- 4Data-Snooping Bias
The multiple-testing problem in strategy research, why repeatedly testing ideas on the same data guarantees false discoveries, and how family-wise error and false-discovery-rate corrections raise the bar.
- 5Overfitting
Fitting a model or strategy so closely to historical data that it fails out-of-sample.
- 6Walk-Forward Analysis
Out-of-sample testing by rolling a training window forward and evaluating on the next period repeatedly.
- 7Cross-Validation for Time Series
Why ordinary k-fold cross-validation is invalid on financial time series, the iid assumption fails, autocorrelation and overlapping labels leak across folds, and shuffling trains on the future.
- 8Purged & Embargoed Cross-Validation
López de Prado's fix for validating models on labeled financial data, purge training observations whose label windows overlap the test set, then embargo a buffer after it to kill serial-correlation leakage.
- 9The Deflated Sharpe Ratio
Bailey & López de Prado's correction that adjusts an observed Sharpe ratio for the number of trials, non-normal returns, and sample length, deflating it against the expected maximum Sharpe achievable by luck alone.
- 10Feature Engineering for Finance
Building predictive features from financial data without destroying signal, the stationarity-versus-memory tradeoff, fractional differentiation, and detecting structural breaks.
- 11Triple-Barrier Labeling
López de Prado's method for turning price paths into supervised-learning labels, a profit-taking barrier, a stop-loss barrier, and a time barrier, plus meta-labeling to separate the side of a bet from its size.
- 12Tree Ensembles in Finance
Random forests and gradient boosting on financial data, why their flexibility overfits low-signal, non-stationary markets, and why their feature-importance measures mislead.
- 13Backtest Overfitting
The probability that the best-performing backtest is in-sample-optimal but out-of-sample worthless, measured by the PBO via combinatorially-symmetric cross-validation, and bounded by the minimum backtest length.
- 14Transaction Costs
Slippage, spread, commission, and market impact that reduce strategy returns in practice.