PnL Concentration and Outlier-Driven Returns
How to check whether a strategy's entire track record is really the accumulated result of many decent trades, or whether it's secretly propped up by a handful of extreme days that a slightly different backtest window would have missed.
Prerequisites: Five-Number Summary and Box Plots, Outlier Detection and Winsorization
Two strategies both show a solid two-year track record with a Sharpe ratio around 1.2. One earned that steadily, a little every week. The other earned essentially all of its profit on four unusually good days, with the remaining 500 days roughly flat or slightly negative. Both have the same headline number, but they are completely different bets going forward — the second strategy's entire performance rests on whether those four rare events recur, and a summary statistic like the Sharpe ratio doesn't tell you which situation you're in.
Measuring concentration
The simplest check is to sort daily (or trade-level) PnL from largest to smallest and ask what fraction of total profit comes from the top handful of days. If the top 5% of days account for 80% of cumulative profit, the strategy is highly concentrated; if they account for something closer to the 5% you'd expect from evenly spread contributions, it isn't — an idea directly analogous to Gini-style inequality measures, applied to PnL instead of income.
A related, more direct check is simply removing the single best day from the backtest and recomputing the Sharpe ratio and total return. If dropping one day turns an attractive track record into a mediocre one, the strategy's apparent edge is a statement about one event dressed up as a statement about a repeatable process. A genuinely diversified strategy should be largely unaffected by removing any single day.
Worked example
A strategy's 500 daily PnLs sum to $100,000 total profit. Sorting them, the best single day contributed $45,000 and the best five days combined contributed $78,000 — meaning the remaining 495 days, taken together, contributed only about $22,000, or roughly $44 per day on average across the bulk of the sample. Removing just that one $45,000 day cuts total profit from $100,000 to $55,000 and would likely cut the reported Sharpe ratio by a similar large fraction, since that day was also probably a large-variance contributor. This strategy's headline numbers are telling a story about one exceptional day far more than about a repeatable process — worth investigating what happened that day (a rare market dislocation the strategy was positioned for by luck, or genuine skill in a rare regime) before trusting the track record to repeat.
What this means in practice
Concentration checks matter most before allocating real capital to a strategy with a short track record, where a handful of days can dominate the entire sample. They also matter when comparing strategies for a portfolio: one whose PnL is concentrated in the same rare events as others you already hold adds much less diversification than its standalone Sharpe ratio suggests. This isn't a verdict that concentrated PnL is bad — tail hedges and event-driven trades are supposed to be concentrated by design — the point is knowing which kind of strategy you're looking at before trusting the summary statistic.
A track record's headline statistics (mean, Sharpe) can look identical for a steadily profitable strategy and one whose entire edge came from a handful of rare days. Check what fraction of total PnL comes from the top few days, and see how much the reported performance changes when the single best day is removed, before trusting a short track record.
Related concepts
Practice in interviews
Further reading
- Bailey & Lopez de Prado, 'The Sharpe Ratio Efficient Frontier', Journal of Risk (2012)