Why Machine Learning Funds Fail: Lopez de Prado's Ten Autopsies

There is a comforting story a failing quant fund can tell itself: we had the right idea, we just needed a better model. Swap the random forest for a neural network, add more layers, get more GPUs, and the alpha will appear.

Marcos Lopez de Prado, who has spent his career building quantitative strategies at large institutions, thinks this story is almost always false. In his account, machine learning funds do not fail because their algorithms are inadequate. They fail because the humans running them are making basic errors that have nothing to do with the algorithms at all.

This paper is his list. It is not a mathematical contribution. It is an autopsy report, and it is one of the most practically useful things a working quant can read, because almost every item on the list is a mistake you can stop making tomorrow.

The problem: financial data breaks the assumptions machine learning was built on

The unifying theme, and it is worth stating before the list, is this.

Machine learning was developed on problems like image recognition and spam filtering. Those problems have properties that financial data does not have: abundant data, high signal-to-noise, independent observations, a stationary world, and no adversary actively trying to destroy your edge.

Financial data has none of those properties. And if you take a toolkit built on those assumptions and apply it, unmodified, to data that violates every one of them, the tools will not fail loudly. They will fail quietly, producing beautiful results that are entirely fictitious. That is the danger. A broken pipeline in finance does not throw an error. It prints a Sharpe ratio of 3.

Here are the failure modes he identifies.

The key idea via analogy: a lone researcher is a lottery machine

Several of Lopez de Prado's points are worth drawing out.

The Sisyphus problem: the way quant teams are organised guarantees overfitting. The standard model is to hire a group of talented individuals and set each of them to work, alone, on discovering a strategy end-to-end: find the data, build the features, fit the model, run the backtest, present the result. Everyone is a soloist.

His argument against this is devastating, and it is a multiple-testing argument in disguise. Fifty researchers each independently searching for a strategy is fifty independent searches, and the one that gets presented to the investment committee is the luckiest of fifty, not the best of fifty. Nobody counted the trials, because nobody thinks of a colleague's failed idea as one of their own trials. But statistically, it is. The organisation is running a lottery and mistaking the winning ticket for skill.

His proposed alternative is a research factory: specialised teams, each responsible for one stage of the pipeline (data curation, feature engineering, strategy construction, backtesting, deployment), with rigorous handoffs. Nobody sees a result end-to-end, so nobody can shop for one.

The backtest is being used as a research tool, and it must not be. A backtest is a test, and a test can only be used once. The moment you run a backtest, look at the result, adjust your strategy, and run it again, you have converted your out-of-sample data into training data. Do this a hundred times and your "out-of-sample" backtest is the most overfit object in the building. He is blunt: a backtest is not a research tool, it is a final check, and if you have looked at it more than a handful of times it is worthless.

Chronological cross-validation is not enough, because of leakage. Everyone knows not to train on the future. Far fewer people realise that even a properly ordered train/test split leaks information in finance. The reason: labels overlap. If your label is "the return over the next twenty days," then the label for Monday and the label for Tuesday share nineteen days of the same future. So a training example from Monday and a test example from Tuesday are not independent. They are almost the same observation. Your test set is contaminated, and your measured performance is inflated.

His fix, developed at length in his book, is purging (remove from the training set any observation whose label overlaps in time with the test set) and embargoing (additionally drop a buffer of observations immediately after the test set, because of serial correlation). Without these, your cross-validation score is a fiction. This is, quietly, one of the most useful practical ideas in the whole of financial machine learning, and almost nobody outside the field does it.

Fixed-time-horizon labelling is a bad way to define what you are predicting. The standard label is "the return over the next N days." But no trader actually trades that way. A real trader has a profit target and a stop loss, and exits when one of them is hit. Labelling by fixed horizon means you are teaching your model to predict something nobody will ever act on. His alternative, the triple-barrier method, labels each observation by which of three barriers is touched first: the profit target, the stop loss, or a time limit. This makes the label match the trading decision.

Non-stationary prices are differenced into uselessness. Machine learning models want stationary inputs. The standard fix is to take returns, that is, to difference the prices. But full differencing throws away all the memory in the series: the level of the price, which carries genuine information about where the asset sits relative to its history, is completely erased. His answer is fractional differentiation: difference by a fractional amount, just enough to achieve stationarity, no more, preserving as much memory as possible.

Every observation is treated as equally informative, and they are not. A day when nothing happened and a day of a market crash appear as one row each in your training data. Sampling by clock time gives equal weight to the uninformative and the informative. Better to sample by activity: take an observation every time a certain amount of volume or a certain number of trades has occurred. This produces data whose statistical properties are far closer to what the models assume.

Why it mattered

• It moved the conversation from algorithms to process. The industry was arguing about neural networks versus boosted trees. Lopez de Prado pointed out that both will fail if your labels overlap, your cross-validation leaks, and your organisation is running an unacknowledged lottery. The bottleneck was never the model.
• It named leakage mechanisms nobody was talking about. Label overlap is a subtle, invisible bug that inflates results while looking completely rigorous. Purging and embargoing are now standard in serious financial machine learning pipelines, and this line of work is why.
• It gave managers a due diligence checklist. "How do you handle overlapping labels?" is a question that separates the people who have thought about this from the people who have not, and it takes about ten seconds to ask.
• It reframed the backtest as a scarce resource. The idea that out-of-sample data is a consumable you burn a little of every time you look at it is a genuinely important discipline, and one that most researchers still resist.

The honest limitations

• The evidence is assertion and experience, not proof. This is not an empirical paper with a controlled study demonstrating that funds following these rules outperform. It is the accumulated judgment of an experienced practitioner. That judgment is worth a great deal, but it is not the same as evidence, and the paper's confident tone can obscure that.
• The research factory is a hard sell, and may cost you something real. Splitting research into specialised assembly-line stages controls overfitting, but it also removes the end-to-end intuition that often generates the best ideas in the first place. Talented researchers tend to hate it. Whether the trade is net positive is genuinely contested.
• Fixing every item on the list still does not give you alpha. The paper is a catalogue of ways to avoid fooling yourself. Not fooling yourself is necessary and nowhere near sufficient. A perfectly rigorous research process applied to a market with no exploitable edge will correctly and rigorously find nothing.
• Some prescriptions are more settled than others. Purging and embargoing are close to unarguable. Fractional differentiation and the triple-barrier method are reasonable ideas with less independent validation, and they are presented with a similar level of confidence.
• The tone invites resistance. Telling an entire industry that it is doing everything wrong is an effective way to be right and an ineffective way to be listened to.

The one-line takeaway

Lopez de Prado argues that machine learning funds fail not because their models are too weak but because their research process leaks information, their labels overlap, their backtests have been looked at a hundred times, and their organisation is running an unacknowledged lottery across dozens of researchers and calling the winning ticket skill.