Deep IV and ML Instrumental Variables

Classic instrumental-variables (IV) estimation exists for one specific problem: you want the causal effect of $X$ on $Y$, but $X$ is entangled with something unobserved that also affects $Y$, so a plain regression of $Y$ on $X$ is contaminated. An instrument $Z$ — something that moves $X$ but has no direct effect on $Y$ except through $X$ — lets you isolate the causal piece. Two-stage least squares does this with two linear regressions: predict $X$ from $Z$, then regress $Y$ on that predicted $X$. Both stages are linear because that's what the classic machinery assumes.

Real instruments and real treatment variables are often not linear functions of each other — an instrument might interact with several other variables in a complicated way to determine treatment intensity, or the causal relationship itself might be sharply non-linear. ML-IV methods (Deep IV is the best-known) swap in flexible machine-learning models for both stages. But you can't just paste a neural network into stage one and a gradient-boosted tree into stage two and call it done: doing that naively lets the second-stage model overfit to whatever quirks the first-stage model's predictions happen to have, producing an estimate that looks precise but is actually picking up noise from stage one, not the true causal relationship. Deep IV's specific fix is to have stage one estimate the full conditional distribution of the treatment given the instrument (not just a point prediction), and have stage two integrate over that distribution — via Monte Carlo sampling — rather than plugging in a single predicted value. This keeps the second-stage model from being handed an artificially clean input it can overfit to.

Worked example

A quant wants the causal effect of order size on price impact, but order size and price impact are both driven by unobserved trader urgency — traders in a hurry submit larger orders and accept worse execution, so a plain regression of impact on order size overstates the size effect. A plausible instrument is a temporary change in a venue's minimum order-size rebate tier, which shifts typical order sizes for reasons unrelated to any individual trader's urgency. Instead of a linear first stage, a neural network models the full distribution of order sizes conditional on the rebate tier and other controls, capturing that the relationship is different for small-cap versus large-cap names. The second-stage model then predicts price impact by averaging over many samples drawn from that first-stage distribution for each instrument value, rather than treating stage one's single "average predicted order size" as if it were the truth — precisely to avoid stage two fitting to first-stage overconfidence.

What this means in practice

ML-IV methods matter wherever a good instrument exists but the relationships involved are visibly non-linear — dose-response curves, price-impact functions, network effects — situations where classic 2SLS would force a straight line onto a curved relationship and misstate the effect. The hard part is almost never the ML machinery; it's finding an instrument that actually satisfies the "affects $Y$ only through $X$" requirement, which is an economic argument, not a statistical one, and no amount of modeling sophistication in either stage substitutes for it.