Neyman Orthogonality

Plug a machine learning model into a causal effect estimate — say, using a random forest to predict outcomes as part of a treatment-effect formula — and you'd expect the forest's own prediction errors to leak into your final effect estimate, biasing it in ways that only shrink slowly as you gather more data, since flexible ML models are typically tuned to minimize prediction error, not to avoid contaminating some downstream causal quantity. Neyman orthogonality is the mathematical property that, when it holds, breaks this expectation: it makes the final estimate's error insensitive, to first order, to small errors in the nuisance ML model, so the ML model's own imperfection barely moves the answer at all.

The idea: first-order insensitivity to nuisance-model error

In causal estimation, the parameter you actually want (the treatment effect, say) is called the target parameter, and everything else the estimation formula needs but doesn't itself care about (an outcome model, a propensity model) is called a nuisance parameter. An estimating equation is Neyman orthogonal if its sensitivity to small perturbations in the nuisance parameter, evaluated at the true nuisance value, is exactly zero:

$$\left.\frac{\partial}{\partial \eta}\, \mathbb{E}\big[\psi(W;\, \theta, \eta)\big]\right|_{\eta = \eta_0} = 0$$

In plain English: $\psi$ is the estimating equation used to pin down the target parameter $\theta$ (the effect), and $\eta$ stands for the nuisance parameters (outcome and propensity models). The equation says: if you nudge the nuisance model slightly away from its true value $\eta_0$, the average estimating equation barely changes — its derivative with respect to that nudge is zero. Practically, this means that even if the ML model used for $\eta$ has estimation error that vanishes slowly (as most flexible ML models do, converging at a rate slower than the classic $1/\sqrt{n}$), that error only enters the final effect estimate squared, which vanishes much faster — often fast enough that the effect estimate itself still achieves the fast, standard $1/\sqrt{n}$ rate of convergence, and standard confidence intervals remain valid.

Worked example: why the DR-learner formula is (nearly) orthogonal

Recall the DR-learner's pseudo-outcome from a doubly robust estimator: it adds a propensity-weighted residual-correction term on top of a plain outcome-model difference. That correction term is precisely what makes the overall estimating equation close to Neyman orthogonal: a naive T-learner-style estimate (outcome-model difference alone, with no correction term) is not orthogonal — a small error in the outcome model translates roughly one-for-one into an error in the estimated effect. Adding the propensity-weighted residual term cancels that first-order sensitivity: differentiating the full doubly robust estimating equation with respect to a small perturbation in the outcome model, the correction term's contribution exactly offsets the direct term's contribution, leaving only second-order (squared) error. This is why doubly robust and double-machine-learning style estimators are built with these particular correction terms rather than the seemingly simpler plug-in difference — the correction terms exist specifically to buy orthogonality.

What this means in practice

Neyman orthogonality is the theoretical justification for why it's safe to plug flexible, imperfect machine learning models (gradient-boosted trees, neural nets) into causal effect formulas at all — without it, the ML model's own estimation error and overfitting would directly bias the causal estimate, and the usual statistical guarantees (valid standard errors, confidence intervals) would fail. It's a property of the formula, not of any specific dataset, so practitioners rely on using estimators already proven orthogonal (like the DR-learner or double machine learning) rather than deriving orthogonality themselves for a new estimating equation.