Matching and Nearest-Neighbour Estimators

A quant wants to know how much a firm's new compliance training reduces trading errors, using historical data where attendance wasn't randomized. One direct approach: for every trader who attended training, find an untrained trader who looks as similar as possible — same tenure, same desk, same historical error rate before training — and simply compare their subsequent error rates. Do this for every trained trader, average the differences, and you have an estimate with no regression model, no assumed functional form, just paired comparisons of look-alikes. This is matching, and its appeal is that it mimics an experiment by hand-constructing the closest available control group for each treated unit.

An analogy: sizing a suit by finding your twin

A tailor with no measuring tape but a large photo album of past customers and how their suits turned out could still size your suit reasonably well by finding the person in the album who looks most like you — similar height, build, shoulder width — and using what worked for them. The fewer dimensions you need to match on and the bigger the album, the better a "twin" you can find. Matching estimators do exactly this for causal questions: rather than fitting a global formula relating measurements to suit size, they find your nearest look-alike in the data and borrow directly from their outcome.

The estimator, one symbol at a time

For each treated unit $i$ with covariates $X_i$, define its matched control as the untreated unit $j(i)$ minimizing some distance $\|X_i - X_j\|$ over all untreated $j$ (nearest-neighbour matching; often on standardized covariates, or on a single scalar via the propensity score, see propensity score methods). The matching estimator of the average treatment effect on the treated is:

$$\widehat{\text{ATT}} = \frac{1}{N_1} \sum_{i: D_i=1} \left( Y_i - Y_{j(i)} \right)$$

In plain English: for every treated unit, find its closest untreated twin, subtract twin's outcome from treated unit's outcome, and average that difference across all treated units. This relies entirely on ignorability (see ignorability and unconfoundedness) holding with respect to the matching covariates $X$ — if some unmeasured trait differs systematically between treated units and their matches, the estimate stays biased no matter how close the observed-covariate match is.

Worked example 1: three matched pairs by hand

Suppose three trainees are matched to three untrained look-alikes on tenure and pre-training error rate:

Pair 1: trainee (tenure 2yr, pre-error 8%) matched to twin (tenure 2yr, pre-error 7%). Post-training error rates: trainee 3%, twin 6%. Difference: −3 points. Pair 2: trainee (tenure 5yr, pre-error 4%) matched to twin (tenure 5yr, pre-error 4%). Post: trainee 2%, twin 4%. Difference: −2 points. Pair 3: trainee (tenure 1yr, pre-error 10%) matched to twin (tenure 1yr, pre-error 11%). Post: trainee 5%, twin 9%. Difference: −4 points.

$$\widehat{\text{ATT}} = \frac{(-3) + (-2) + (-4)}{3} = -3 \text{ percentage points}.$$

Training is estimated to reduce error rates by about 3 percentage points on average, among trainees, based entirely on nearby look-alike comparisons — no regression coefficients involved.

Worked example 2: when matches are poor, the estimate breaks down

Suppose a fourth trainee has tenure 8 years and pre-error rate 2% — a very experienced, already-accurate trader who happened to attend training. The nearest available untrained look-alike only has tenure 4 years and pre-error rate 5%, because no untrained trader in the data is nearly as tenured or accurate. This is a poor match: the "twin" isn't actually similar, so their outcome difference reflects tenure and baseline skill gaps as much as training. If trainee's post-error is 1.5% and the poorly-matched twin's is 4%, the pair contributes a difference of −2.5 points that's contaminated by the covariate gap, not a clean estimate of training's effect. This is exactly the overlap (positivity) problem from ignorability — for very experienced, low-error traders, there's essentially no comparable untrained control group, and no amount of matching machinery fixes a missing comparison group.

What this means in practice

Matching is most useful when you want a transparent, model-free estimate and can inspect the actual pairs formed — a genuine advantage over regression, where a poorly-fitting model can silently extrapolate without anyone noticing. Its weakness is the same overlap problem every observational method faces: matching can only be trustworthy where the treated and untreated populations genuinely overlap in covariate space, and it should always be reported alongside match-quality diagnostics (how close were the pairs, on average?), not just the final ATT number. In practice it's often combined with reweighting methods like inverse probability weighting or doubly robust estimation to hedge against matching-model misspecification.