The Potential Outcomes Framework

"Correlation isn't causation" is a slogan everyone knows and almost no one can define precisely. What, exactly, would it mean for a trading signal to causally improve returns, as opposed to merely correlating with them? The potential outcomes framework gives the precise answer, and once you have it, the reason correlation fails becomes a matter of simple arithmetic rather than a vague warning. The core idea: define a causal effect as the difference between two versions of the same unit's future — one where it received the treatment, one where it didn't. You can only ever observe one of those two versions for any given unit. Everything hard about causal inference is a consequence of that one missing half.

An analogy: the road not taken

Imagine you take a new highway to work one morning and arrive in 25 minutes. To know whether the highway actually caused you to arrive faster, you'd need to know how long the old route would have taken you that same morning — same traffic, same weather, same version of you. You can't rerun that morning. Maybe the old route would have taken 30 minutes that day (highway helped), or maybe it would have taken 20 minutes because of some coincidence unrelated to the road (highway hurt, but you'd never know from the one trip you took). Every causal question has this same structure: a treated outcome you observe, and an untreated outcome for the exact same unit at the exact same moment that you can never observe, because time only runs once. This unobservable twin is the counterfactual, and formalizing it is the entire content of the potential outcomes framework.

The framework, one symbol at a time

For unit $i$ (a trader, a stock, a customer) and a binary treatment (received a signal or not), define two potential outcomes: $Y_i(1)$, the outcome unit $i$ would have if treated, and $Y_i(0)$, the outcome unit $i$ would have if not treated. The individual treatment effect is

$$\tau_i = Y_i(1) - Y_i(0) .$$

In words: $\tau_i$ ("tau sub i") is the true causal effect for unit $i$ specifically — the difference between its treated and untreated futures. The fundamental problem of causal inference is that you only ever observe one of the two:

$$Y_i^{obs} = D_i\, Y_i(1) + (1-D_i)\, Y_i(0) .$$

In words: $D_i$ ("D sub i") is 1 if unit $i$ actually received treatment and 0 if not; the observed outcome $Y_i^{obs}$ picks out whichever potential outcome actually happened, and the other one is permanently unobserved for that unit. Since $\tau_i$ itself is unobservable for any individual, research settles for the average treatment effect,

$$\text{ATE} = E[Y_i(1)] - E[Y_i(0)] .$$

In words: average the treated potential outcome across the whole population, average the untreated potential outcome across the whole population, and subtract — a population-level quantity that's estimable even though no individual $\tau_i$ ever is. The naive comparison people actually run — average outcome among the treated minus average outcome among the untreated — equals

$$E[Y \mid D=1] - E[Y \mid D=0] = \text{ATE} + \underbrace{E[Y_i(0)\mid D=1] - E[Y_i(0)\mid D=0]}_{\text{selection bias}} .$$

In words: the naive difference in observed averages equals the true average treatment effect PLUS a bias term measuring whether the people who happened to get treated would have had different outcomes anyway, even without treatment, compared to the people who didn't. This is the precise statement of "correlation isn't causation": the naive comparison is only unbiased when that selection term is exactly zero.

Worked example 1: selection bias in a trading-signal comparison

You want to know if a "buy the signal" flag causally improves next-day returns. Among 100 stocks flagged (treated), average next-day return is $0.80\%$. Among 900 unflagged stocks (untreated), average next-day return is $0.10\%$. Naive comparison: $0.80\% - 0.10\% = 0.70\%$ — looks like a strong effect. But suppose the flag disproportionately fires on stocks that already had unusually strong recent momentum, and momentum alone (with no signal at all) tends to continue at, say, $0.55\%$ average next-day return for stocks with that momentum profile, versus $0.10\%$ for the rest. Then the selection-bias term is $E[Y(0)\mid D=1] - E[Y(0)\mid D=0] = 0.55\% - 0.10\% = 0.45\%$ — flagged stocks would have outperformed by 0.45% even with no signal effect at all, purely because of which stocks get flagged. Subtracting it out: true $\text{ATE} \approx 0.70\% - 0.45\% = 0.25\%$. The naive comparison overstated the signal's real causal value by nearly threefold.

Worked example 2: why randomization fixes it, with numbers

Now suppose instead you A/B test the signal: among 1,000 stocks, you randomly flag 100 for the signal-driven action and leave 900 untouched, regardless of momentum. Because the assignment is a coin flip independent of each stock's characteristics, the treated group and untreated group have, in expectation, identical distributions of $Y_i(0)$ — a flagged stock's untreated potential outcome is drawn from the same population as an unflagged stock's, purely by construction of the randomization. So $E[Y_i(0) \mid D=1] = E[Y_i(0)\mid D=0]$ exactly, the selection-bias term is zero by design, and the naive comparison $E[Y\mid D=1]-E[Y\mid D=0]$ is then an unbiased estimate of the true ATE — no correction needed. If this randomized test produced flagged average $0.35\%$ and unflagged average $0.10\%$, the honest estimate of the signal's causal effect is simply $0.35\%-0.10\%=0.25\%$, matching (as it should) the corrected number from example 1 — randomization is what makes the naive comparison trustworthy in the first place.

Even a modest correlation like the one in the explorer above could be entirely selection bias rather than a causal effect — the scatter alone can never distinguish the two; only knowing how the "treatment" (say, membership in one group versus another) was assigned can.

What this means in practice

• A/B tests and randomized experiments are prized precisely because they zero out the selection-bias term by construction — it's the only design where the naive, easy-to-compute comparison is also the causally correct one.
• Observational trading research almost always has this problem. Comparing returns of stocks that "chose" to have some characteristic (high short interest, analyst coverage, insider buying) to those that didn't is comparing groups that likely differ in $Y_i(0)$ too — methods like fixed effects, instrumental variables, and matching are all attempts to approximate what randomization would have given you.
• The ATE can hide important heterogeneity. A signal might have $\tau_i > 0$ for small-caps and $\tau_i < 0$ for large-caps, averaging to a modest positive ATE that misrepresents the effect for either subgroup — always check whether the treatment effect itself varies across the population, not just its average.
• "No selection bias" is an assumption, never something you can verify directly from the data you have, precisely because $Y_i(0)$ for the treated units is fundamentally unobserved — the best you can do is argue for it (via randomization) or control for observable confounders and hope the unobservable ones don't matter.