Tracking Queue Position in a Simulator

A passive strategy posts a resting buy order at the current best bid and, in the backtest, is marked filled every time the market subsequently trades at or through that price — a simulator assumption that's roughly right if the order arrived first in a nearly empty queue, and roughly nonsense otherwise. Backtested this way, the strategy shows a 95% fill rate on its resting orders. Traded live, the fill rate is 41%. The gap isn't a mystery once you notice what the simulator never modeled: at any real price level there is already a queue of other resting orders ahead of a newly arriving one, and a trade at that price fills the front of the queue first. An order can sit at the correct price for the entire life of a quote and still get nothing, because everyone ahead of it got there first and the trading that occurred wasn't enough to reach it.

This is queue position, and it's the single biggest gap between a naive limit-order backtest and reality for any strategy that posts passive orders rather than crossing the spread to trade immediately.

What determines whether you fill

Two things move an order toward the front of the queue: trades occurring at that price level (which consume the queue from the front, oldest order first) and cancellations by orders ahead of you (which remove them from the queue without a trade happening at all — a meaningful share of order flow, since many resting orders get pulled rather than filled). Your fill probability over some horizon is roughly a race between "enough volume trades through this level to reach my position" and "the price moves away before that happens," and queue position is what sets how much volume "enough" requires.

A simulator that wants to get this right has to track, for a simulated order joining a price level, exactly how much size was resting ahead of it at the moment it joined, then deplete that ahead-size as historical trades and cancels occur at that level, and only mark the simulated order filled once the ahead-size reaches zero and there's still incoming volume left to reach it.

Worked example: two orders, same price, different odds

A price level at $40.02 has 20,000 shares resting when strategy order X joins at the back, making the queue 20,000 shares deep ahead of X. Over the next minute, market sell orders trade 14,000 shares through that level (consuming the queue from the front) and 4,000 shares of resting orders ahead of X get cancelled. Ahead-size for X has dropped from 20,000 to 20,000 − 14,000 − 4,000 = 2,000. X is not filled yet — 2,000 shares still stand between it and a trade — but it has moved from position 20,000 to position 2,000 in the queue, meaningfully increasing its odds of filling in the next minute.

Compare that to a second order, Y, that joined the same price level five seconds before X, when only 6,000 shares were resting ahead of it. Y's ahead-size after the same minute of activity is max(0, 6,000 − 14,000 − 4,000) = 0 — Y has already filled, likely partway through that minute, while X is still waiting. Same price, same minute, same market activity: Y is filled and X is not, purely because of five seconds of arrival order. A simulator that only checks "did the price trade here" cannot tell X and Y apart at all.

Doing it properly

Reconstruct queue position from the most granular data available — full order-by-order message logs where possible, since anything coarser is an approximation of an approximation. Track cancellations ahead of your order as carefully as trades, since in many markets cancels remove more resting size from a queue than trades do. Calibrate the resulting fill-rate model against your own live fills whenever you have them — see Calibrating a Simulator Against Live Fills — because even a queue-aware simulator carries assumptions (how cancels are distributed within the queue, how your own order might affect others' cancel decisions) that only real fills can validate.