Off-Policy Evaluation of Trading Policies

Backtesting a trading strategy usually assumes you can just replay history and simulate what the strategy would have done, since prices and fills don't depend much on what a small strategy does. That assumption breaks down for execution policies and market-making strategies whose own trades measurably move prices and consume liquidity — replaying historical prices doesn't tell you what would have happened to fills and prices if the new policy had actually been trading, because the old policy's trades are baked into the historical data you're replaying. Off-policy evaluation for trading policies is the set of techniques for estimating a new policy's performance from logs generated by an old policy, accounting for the fact that the new policy would have acted differently and therefore experienced a different market.

The mechanics mirror off-policy evaluation elsewhere: reweight the logged outcomes according to how much more or less likely the new policy would have been to take each logged action compared to the old policy that actually generated the data, so states and actions the new policy favors count for more in the estimate. In execution specifically, this often has to be combined with a market-impact model, since a new execution policy trading larger or smaller size than the old one would have faced different slippage than the historical logs show — reweighting the logged outcomes alone isn't enough if the actions themselves would have changed the market being traded in.

For example, a firm wants to evaluate whether a new execution algorithm that trades more passively than its current one would reduce costs, without risking client orders on an untested algorithm. Off-policy evaluation reweights historical execution logs toward the situations where the new, more-passive policy would have acted similarly to the old one, and combines that with a market-impact adjustment for situations where the new policy's smaller order sizes would have moved the market less than the logged trades did — producing a cost estimate for the new policy without ever running it live.

What this means in practice

This is the standard way execution and market-making desks get comfortable trying a new RL-based policy before committing real order flow to it, since a live pilot is expensive if the new policy turns out worse. The reliability of the estimate depends heavily on how different the new policy's actions are from the old one — small changes can be evaluated fairly confidently, but a radically different policy pushes the technique past what reweighting historical data can trustworthily support, and eventually requires a live test regardless.