Simpson's Paradox

A quant compares two trading strategies, A and B. In large-cap stocks, A beats B. In small-cap stocks, A also beats B. Pool everything together across the whole portfolio, and B beats A. Nothing was miscalculated — both statements can be simultaneously, arithmetically true. This is Simpson's paradox, and it isn't really a paradox at all once you see the mechanism: it's what happens when the mix of large-cap versus small-cap trades differs between the two strategies, and that mix itself matters for the outcome.

An analogy: two batters, two seasons

A classic version: batter X has a higher batting average than batter Y in the first half of the season, and again in the second half — yet Y has the higher average over the full season. This happens when X and Y face very different numbers of at-bats in each half. If X had far more at-bats in the half where averages are generally lower (say, against tougher pitching), X's full-season average gets dragged down by that heavily-weighted, low-average half, even while beating Y in both halves individually. The paradox isn't in the math — it's the reminder that a weighted average is not the same as "true in each part, therefore true overall."

The mechanism, one symbol at a time

Let $D$ be strategy choice (A or B), $Y$ be win/loss outcome, and $Z$ a variable — like trade size category — that correlates with both which strategy handled a trade and how likely that trade was to win. Within each stratum of $Z$:

$$P(Y=\text{win} \mid D=A, Z=z) > P(Y=\text{win} \mid D=B, Z=z) \quad \text{for every } z,$$

yet pooling across strata with different weights $P(Z=z \mid D=A) \ne P(Z=z \mid D=B)$ can flip the marginal inequality:

$$P(Y=\text{win} \mid D=A) < P(Y=\text{win} \mid D=B).$$

In plain English: A wins more often than B within every category, but if A was disproportionately applied to the harder category and B to the easier one, A's overall record gets dragged down by that unfavorable mix, even though its performance category-by-category was uniformly better.

Worked example 1: the two strategies by hand

Strategy A: large-cap trades, 90 out of 100 win (90%); small-cap trades, 20 out of 100 win (20%). Strategy B: large-cap trades, 80 out of 100 win (80%); small-cap trades, 10 out of 50 win (20%)... let's use numbers that actually reverse. Strategy A: large-cap 81/90 win (90%), small-cap 20/100 win (20%) — total 101/190 ≈ 53.2%. Strategy B: large-cap 8/10 win (80%), small-cap 90/150 win (60%) — total 98/160 ≈ 61.3%. A beats B in large-cap (90% > 80%) and in small-cap (20% < 60% — wait, check per-category). Adjust cleanly: A wins large-cap 90% vs B's 80% (A ahead), and A wins small-cap 30% vs B's 20% (A ahead), but A only ran 10 large-cap trades and 190 small-cap trades, while B ran 190 large-cap and 10 small-cap. A's overall = (0.90×10 + 0.30×190)/200 = (9+57)/200 = 33%. B's overall = (0.80×190 + 0.20×10)/200 = (152+2)/200 = 77%. A wins every category yet loses overall, purely because A's volume was concentrated in the weaker category and B's in the stronger one.

Worked example 2: reading a bank's loan approval data

A bank's loan approval rate is higher for men than women overall, 60% vs 50%. Broken down by income bracket, women have a higher approval rate than men in both the low-income bracket (30% vs 25%) and the high-income bracket (85% vs 80%). The reversal happens because far more of the male applicants fall in the high-income bracket (where approval rates are high for everyone) and far more female applicants fall in the low-income bracket (where approval rates are low for everyone). The correct, decomposed comparison (women outperform men within every income bracket) tells a very different, arguably more important story than the pooled headline number, and only the within-bracket comparison is useful for judging whether the approval process treats men and women fairly conditional on income.

What this means in practice

Simpson's paradox shows up constantly in performance attribution, A/B tests with unbalanced traffic allocation, and any report that pools heterogeneous subgroups. The fix is not a statistical trick — it's asking what variable determines the group sizes and whether that variable belongs in the analysis as a confounder (see confounding and the backdoor criterion). If it's a genuine common cause of both group assignment and outcome, report the stratified, within-group comparison, not the pooled one; if instead it's a downstream consequence of treatment, pooling may be correct after all, and stratifying could introduce a collider bias of its own. See Simpson's paradox trap for a walkthrough of that judgment call.