Simulator Fidelity Versus Speed

A research team wants to search over 10,000 parameter combinations for a passive limit-order strategy. A full limit-order-book simulator that replays every message and tracks true queue position takes 40 minutes per run — three weeks of compute for the full sweep. So they build a simplified simulator: fills happen with a fixed probability whenever the touch price crosses the order, no queue, no message-level detail. The sweep finishes overnight, the best configuration looks excellent, and it's shipped. Live, the strategy fills at less than half the rate the simplified simulator predicted, because the simplified model never captured that passive orders sit behind a queue of other orders that have to cancel or fill first — the exact mechanic that determines whether a resting order gets touched at all. See Tracking Queue Position in a Simulator.

This is the fidelity-speed tradeoff, and it isn't a bug in either simulator — it's the central design decision of building one at all. A higher-fidelity simulator, one that models the order book message by message, gets closer to how the real market actually behaves, at the cost of being slow enough that you can only afford to run it a few times. A lower-fidelity simulator runs fast enough to search a huge parameter space, at the cost of getting the mechanics wrong in ways that systematically favor whichever strategies happen to exploit its specific simplifications.

Why the fast simulator doesn't fail randomly

The danger isn't that a fast simulator is noisy — it's that its errors are biased, and a parameter search actively hunts down and exploits that bias. If a simplified fill model overstates fill probability for passive orders, a sweep over thousands of configurations won't just tolerate that overstatement, it will preferentially select the most aggressive passive strategies, the ones that lean hardest on the very assumption that's wrong. The search doesn't know it's exploiting a modeling gap; it just sees a better Sharpe ratio and picks it. The more parameter combinations you search, the more thoroughly you find and exploit whatever the fast simulator gets wrong — the same mechanism as overfitting to backtest noise, except here the "noise" is a structural bias in the simulator itself, not sampling variation in the data.

Worked example: same strategy, two simulators, two answers

A passive market-making variant is evaluated both ways. The simplified simulator assumes a resting order fills whenever the mid price touches its level, independent of how many orders were ahead of it in the queue:

Simulator	Assumed fill rate	Backtested Sharpe	What it's missing
Simplified (no queue)	85%	2.4	Queue position, order cancellations ahead of you
Full order-book replay	38%	0.9	(models it directly)

The full simulator's 38% isn't just "lower" — it's the number that matches what live trading actually produced when the strategy was eventually tested with real orders. The simplified simulator wasn't wrong by a little; it was wrong by more than a factor of two, and a parameter sweep run on it would have ranked strategies by how aggressively they leaned on that gap, not by how good they actually were.

Doing it properly

Match simulator fidelity to what you're deciding: a first-pass screen over thousands of parameter combinations can tolerate a simplified model if the final candidates get re-checked at full fidelity, but a capacity or capital-allocation decision should never rest on the fast simulator alone. Periodically calibrate the fast simulator's key assumptions — fill probability, spread, impact — against what the full simulator or live fills actually produce, so its bias is at least a known, bounded quantity rather than an unknown one. See Calibrating a Simulator Against Live Fills for how that calibration loop works in practice, and treat a large gap between the two simulators' answers as a research finding in its own right, not just an inconvenience to average away.

It also helps to budget compute deliberately rather than defaulting to whichever simulator happens to be already wired up: decide up front how many candidates a search needs to survive to before full-fidelity validation becomes affordable, and design the search in stages so the expensive simulator only ever sees a shortlist. A two-stage process — a wide, fast screen followed by a narrow, slow confirmation — usually costs a fraction of running everything at full fidelity while catching the bulk of the bias a fast-only search would have exploited. Keep a running record of how often the two simulators disagree on the final ranking of a shortlist, since a search that keeps agreeing gives you real confidence to lean more on the fast simulator next time, while one that keeps disagreeing is telling you the fast model's assumptions need another calibration pass before they're trusted again.