Policy Learning and Optimal Treatment Rules

Knowing that a marketing campaign raised average conversion by two percentage points doesn't tell you which customers to send it to. Some customers would have converted anyway; some won't convert no matter what; some are on the fence and the campaign is exactly what tips them. Policy learning is the step past estimating an average effect: it asks who, specifically, should get the treatment, given that treating everyone (or no one) is rarely optimal once treatment has a cost.

A treatment rule is a function from a unit's observed characteristics to a decision — treat or don't, for a customer, patient, borrower, or trade. An optimal policy is the treatment rule that maximizes some objective, typically total benefit minus total cost of treating.

Why this needs its own machinery

You might think: estimate each individual's treatment effect, then treat everyone whose estimated effect is positive. This is close to right but has a subtle problem — a model trained to predict outcomes accurately is not the same as a model trained to make good treatment decisions, because prediction error that's small on average can still flip the sign of an estimated effect for units near the decision boundary, which are exactly the units the policy decision is most sensitive to. Policy learning methods instead often optimize the decision rule directly against the actual objective — total welfare or profit under the rule — rather than optimizing predictive accuracy and hoping a good decision rule falls out.

Uplift modelling — predicting each unit's individual treatment effect and thresholding it — is a common practical version of this idea; see Uplift Modelling. Policy learning is the more general framing that also covers rules constrained in ways a simple threshold can't express, such as "treat at most 20% of the population" or "the rule must depend on only a few interpretable features."

Worked example (light): a lending offer

A bank wants to know which customers should receive a lower interest-rate offer to reduce churn. Estimated individually, the effect of the offer on retention is positive for most customers but the offer itself costs the bank margin. A policy rule built directly against "maximize retained margin" — not against "maximize predicted retention" — will exclude customers whose estimated retention gain is real but too small to cover the margin given up, even though a naive "give everyone with a positive effect the offer" rule would include them.

Where it's used and where it breaks

Policy learning shows up anywhere a costly action is applied selectively: which accounts get a compliance review, which positions get a hedge overlay, which customers get a retention offer. It requires the same causal-inference care as any treatment-effect estimate — the training data has to reflect real variation in who was treated, ideally from a randomized or well-controlled source, or the learned policy just reproduces whatever bias determined treatment historically.

Constraints matter as much as the objective

Real policies are rarely "treat whoever has a positive estimated effect" — they're constrained by budget, capacity, or fairness requirements that change what "optimal" even means. A compliance team can only review so many accounts per week regardless of how many would benefit from review; a retention-offer budget can only fund so many discounts. Policy learning under a budget constraint asks for the subset of a fixed size that maximizes total benefit, which is a different and generally harder problem than simply thresholding an individual effect estimate, because the right cutoff depends on the whole distribution of estimated effects, not on any single unit in isolation. Some settings add a fairness or interpretability constraint on top — the rule must not depend, even indirectly, on a protected characteristic, or it must be simple enough that a regulator or a customer can be told the actual criterion. Both change the optimization problem, not just its interpretation, and a policy learned while ignoring them can be unusable however well it scores on the underlying objective.

Evaluating a learned policy honestly

Because the policy decides who gets treated, you can't simply check its outcomes against the historical data it trained on — that data mostly describes what happened under the old assignment rule, not the new one. The standard approach is off-policy evaluation: reweighting historical outcomes by how likely the old policy was to have taken the same action the new policy would recommend, so that units the new policy would treat get counted based on what actually happened to similar units that were, by chance, also treated historically. This only works if the historical data has decent coverage of both treating and not treating similar units — a policy trained on data where treatment was applied almost deterministically by some existing rule has very little honest evidence to evaluate against, and any off-policy estimate built from it should be treated with real caution regardless of how confident the resulting number looks.