Off-Policy Evaluation of Decision Policies

Suppose a firm's current policy for approving loan applications, offering discounts, or routing customer complaints was built years ago, and a data scientist has designed a new policy that looks better on paper. Testing it properly would mean actually running it — approving or denying real applications differently — which is expensive, risky, and slow if the new policy turns out to be worse. Off-policy evaluation is a set of statistical techniques for estimating how well the new policy would perform, using only historical data logged under the old policy, without ever deploying the new one.

The core difficulty is that the logged data only shows what happened under the old policy's choices — if the old policy never approved a certain type of borderline application, there's no direct data on what would have happened had it been approved. The standard fix is importance weighting: reweight each logged outcome by how much more (or less) likely the new policy would have been to make that same choice compared to the old policy, so that outcomes the new policy would favor count for more and outcomes it would avoid count for less. A "doubly robust" version combines this reweighting with a separate outcome model, so the estimate stays reasonably accurate even if one of the two pieces is somewhat wrong.

A simple illustration: an old policy offers a discount to 30% of customers, chosen somewhat randomly, and logs which of those converted. A new policy would offer the discount to a different 40% of customers, overlapping partially with the old policy's choices. Off-policy evaluation reweights the logged outcomes — upweighting customers both policies would have targeted, downweighting customers only the old policy targeted — to estimate the new policy's conversion rate, without ever actually running it.

What this means in practice

Off-policy evaluation is what lets a firm iterate on decision policies — credit, marketing, trading execution rules — quickly and cheaply using historical logs, reserving expensive live A/B tests for policies that already look promising offline. Its accuracy depends heavily on the old policy having explored enough different choices in the past; if the old policy was too deterministic, there may simply be no historical data covering what the new policy would do, and no reweighting trick can invent it.