Fill Simulator Fidelity for Training Execution Models

Reinforcement learning and other trial-and-error methods for execution algorithms need to try actions and see outcomes many times, and doing that on live markets — actually placing millions of exploratory orders to learn from the results — is neither affordable nor safe. That pushes execution research toward simulation: a model of the market that can be queried "if I place this order here, what would happen?" But a policy is only ever as good as the simulator it was trained or evaluated in, and a simulator that gets the mechanics of fills wrong will teach a model to exploit mistakes in the simulator rather than to trade well in real markets.

Where simplified simulators break

The simplest fill simulator assumes a static book: place a limit order, and it fills whenever the recorded historical trade tape shows a trade at or through that price, as if the order joined the queue instantly with no effect on anyone else. This ignores queue position — the historical tape reflects trades that happened without your order there, so a naive simulator will happily fill an order the instant a matching historical trade occurs, even though in reality the order would have had to wait behind everyone already resting ahead of it in the queue. It also ignores that placing an order changes the book slightly (adding your own size can itself affect whether others trade at that level) and that canceling and reposting takes real time, during which the market can move.

Queue-aware simulation

A higher-fidelity simulator tracks queue position explicitly: when a new order is placed, it's inserted behind the recorded resting size at that price level at that instant, and it only "fills" once the historical tape shows enough volume trading through that level to have consumed everyone ahead of it first. This single change dramatically affects how attractive passive orders look in simulation — a naive simulator systematically overstates how often passive orders fill (since it ignores the queue entirely), which in turn teaches a learned policy to be too passive, chasing fills that wouldn't actually have happened at the modeled speed.

Worked example: the cost of ignoring the queue

A backtest of a passive limit-order strategy using a naive tape-matching simulator reports a 65% fill rate and strong net P&L, because any historical trade at the limit price counts as a fill regardless of what else was resting there. Rerunning the identical strategy through a queue-aware simulator — which requires the historical volume traded through that level to first exceed the size resting ahead of the strategy's order — drops the fill rate to 38%, and with fewer of the "good" fills where price didn't move away, the reported edge nearly disappears. The strategy didn't change at all; only the simulator's honesty about queue mechanics did, and that alone was enough to turn an apparently profitable backtest into a marginal one.

What this means in practice

Any execution policy — whether it's a hand-tuned rule or a learned model — that looks good in a naive simulator needs to be re-validated in a queue-aware one before anyone trusts the result, since the two can disagree by enough to flip a strategy from profitable to unprofitable. Beyond queue position, higher-fidelity simulators also need to model that other market participants would likely have reacted differently had the strategy's own orders actually been present in the historical market — a much harder problem addressed by counterfactual evaluation and agent-based simulation, since real markets aren't a fixed, unreactive backdrop.