Positivity and Overlap Violations

Every causal comparison from observational data leans on an assumption that's easy to state and easy to overlook: for every combination of covariates in your data, there had to be some chance of seeing either treatment or control. If, within some slice of the population, one arm simply never happens, there is no comparison to make there — you're not estimating a treatment effect, you're extrapolating a model's guess about a group you've literally never observed under the other condition. That requirement is called positivity (or overlap), and it fails constantly in trading and business data because assignment is rarely random — it follows a rule.

Overlap is easy to check informally: plot the estimated propensity score for the treated group and the control group on the same axis. Good overlap looks like two histograms that mostly sit on top of each other across the range. A violation looks like two histograms that barely touch — the treated group clusters near propensity 1, the control group clusters near propensity 0, and there's a stretch of covariate space where you essentially never see a control unit that looks like a treated one, or vice versa.

Worked example

A brokerage always routes orders above 10,000 shares to a block-trading desk and orders below that size to the open market — a deterministic rule, not a coin flip. If you try to estimate the causal effect of "using the block desk" on execution cost by comparing outcomes across order size, you'll find that for orders around 15,000 shares the propensity of using the block desk is essentially 1, and for orders around 2,000 shares it's essentially 0. There simply isn't a "control" observation resembling a 15,000-share order routed to the open market to compare against — any comparison in that region relies entirely on the model extrapolating what would happen, not on data that shows it. The honest fix isn't a better estimator; it's restricting the analysis to the order-size range where both routing outcomes genuinely occur, and being explicit that the effect estimate simply doesn't generalize outside that range.

What this means in practice

Overlap violations don't announce themselves — a doubly-robust estimator or an ML propensity model will still output a number, and it will look exactly like a normal treatment-effect estimate. Checking the propensity distribution before trusting the estimate is not optional diagnostic hygiene, it's the difference between measuring an effect and inventing one. When overlap fails, the standard responses are to trim the sample to the region of genuine overlap, restate the estimand as applying only to that region, or fall back to a design (an experiment, a natural instrument) that doesn't require it in the first place.