Inverse Probability Weighting

Suppose only 10% of low-volatility periods but 60% of high-volatility periods saw a new hedging overlay applied, because traders reached for it mainly when things got scary. A naive comparison of "returns with overlay vs. without" mostly compares high-vol periods (with overlay) to low-vol periods (without) — the wrong comparison entirely. Inverse probability weighting fixes this without discarding any data or building an outcome model: it just up-weights the rare combinations (like an overlay used in a calm period) and down-weights the common ones, so that after reweighting, every period counts as if overlay use had been assigned independently of volatility.

An analogy: correcting a lopsided poll

If a phone survey oversamples retirees (who answer calls more) and undersamples young adults, but you know the true population shares, you can correct the poll's average by giving each retiree response less weight and each young-adult response more weight, in inverse proportion to how over- or under-represented their group was in your sample. The corrected, reweighted average reflects the true population, even though you never collected a single extra response — you just adjusted how much each existing response counts. IPW applies exactly this logic to treatment assignment instead of survey sampling: reweight so treated and untreated groups look, on average, like they were assigned by a coin flip.

The estimator, one symbol at a time

Let $e(X_i) = P(D_i = 1 \mid X_i)$ be the propensity score — the probability unit $i$ receives treatment, given its covariates. Weight each treated unit by $1/e(X_i)$ and each untreated unit by $1/(1-e(X_i))$:

$$\widehat{\text{ATE}} = \frac{1}{N}\sum_{i=1}^{N} \left( \frac{D_i Y_i}{e(X_i)} - \frac{(1-D_i) Y_i}{1-e(X_i)} \right)$$

In plain English: a treated unit that was unlikely to be treated (small $e(X_i)$) gets a large weight, because it's standing in for many similar units that could have been treated but weren't; an untreated unit that was unlikely to go untreated gets a large weight for the mirror reason. Rare-but-informative combinations get amplified, common-and-redundant combinations get shrunk, and the weighted average behaves like a comparison from a randomized experiment — provided the propensity score model is correctly specified and ignorability holds.

Worked example 1: the hedging overlay by hand

Suppose in high-vol periods, $e(X) = 0.6$ (60% chance of overlay use); in low-vol periods, $e(X) = 0.1$. Consider one low-vol period where the overlay was used (rare event) with realized return +2%. Its weight is $1/e(X) = 1/0.1 = 10$ — it counts ten times over, standing in for the many similar low-vol periods where overlay wasn't used but plausibly could have been. Compare to a high-vol period where overlay was used, return −1%: weight $1/0.6 \approx 1.67$, counted less heavily since overlay use was already common in that regime. Averaging across all periods with these weights (rather than raw counts) prevents the analysis from being dominated by the common combinations (high-vol-with-overlay, low-vol-without) and gives proper voice to the rarer, more informative combinations.

Worked example 2: seeing the reweighting balance out

Say the raw data has 100 high-vol periods (60 with overlay, 40 without) and 100 low-vol periods (10 with overlay, 90 without). Raw comparison: average return "with overlay" mixes 60 high-vol + 10 low-vol observations, heavily tilted toward high-vol; average "without overlay" mixes 40 high-vol + 90 low-vol, heavily tilted toward low-vol — these two averages are being computed over different populations, not a fair comparison. After IPW, each of the 60 high-vol-with-overlay periods counts $1/0.6 \approx 1.67$ (total weight $\approx 100$), each of the 40 high-vol-without counts $1/0.4 = 2.5$ (total weight $= 100$), each of the 10 low-vol-with counts $1/0.1 = 10$ (total weight $=100$), each of the 90 low-vol-without counts $1/0.9 \approx 1.11$ (total weight $\approx100$). Every treatment-by-regime cell now contributes the same total weight (100), so the weighted "with" and "without" groups are balanced across volatility regimes — exactly mimicking what a randomized 50/50 assignment within each regime would have produced.

What this means in practice

IPW is a core building block behind more robust estimators and is directly useful whenever the propensity score is easier to model reliably than the outcome itself — for instance, when treatment assignment follows a known rule (like a volatility-triggered policy) but outcomes are noisy and hard to model directly. Its Achilles' heel is extreme weights: when $e(X_i)$ gets very close to 0 or 1 for some units, their weights explode, and a handful of observations can dominate and destabilize the whole estimate. In practice, analysts trim or stabilize weights and check for adequate overlap before trusting an IPW estimate. Doubly robust estimation combines IPW with an outcome model so that even if one of the two is misspecified, the estimate can still be consistent.