Fill Prediction: Features, Labels and Evaluation

An execution algorithm deciding whether to rest a limit order at the best bid or to cross the spread with a marketable order needs an answer to one practical question: "if I place this passive order here, what's the chance it actually gets filled before the price moves away or before I need the shares?" A fill-prediction model answers exactly that, and getting it right depends more on how carefully the label and features are defined than on which algorithm does the fitting.

Defining the label without cheating

The obvious label — "did this order eventually fill, yes or no?" — is dangerously incomplete, because an order that never fills because the trader canceled it after five seconds is not the same event as an order that never fills because it sat for an hour without a match. The standard fix is to condition on a fixed horizon: did the order fill within the next $H$ seconds (or before the price moved a certain distance away), given that it was still live at the start of that window? This turns fill prediction into a well-posed binary classification problem with a clear, comparable target across orders of different actual lifetimes, rather than conflating "didn't fill" with "was withdrawn before it had a chance to."

Features: queue position dominates

The single most informative feature for a resting order is its queue position — how much size sits ahead of it at the same price level, since price-time priority means everyone ahead must trade or cancel before this order gets touched. After that, useful features include the order's distance from the current mid (an order at the best bid fills far more often than one two ticks back), recent trade arrival rate at that price level, and order-book imbalance (heavy buying pressure predicts the ask side filling faster). A common mistake is using the size of the order itself as a feature without also capturing what's ahead of it — a 100-share order that's third in a 50,000-share queue behaves nothing like a 100-share order that's first in an empty queue, even though both are "100 shares."

Worked example: same order, two queue positions

Two limit buy orders of identical size are placed at the best bid at the same instant, one instant apart, in a book where sell-side pressure is picking up. Order A has 2,000 shares ahead of it in the queue; order B, placed a few levels back with far less size ahead, has only 200 shares ahead. Over the next 10 seconds, historical data shows orders with under 500 shares ahead fill about 70% of the time in similar book conditions, versus about 15% for orders with over 1,500 shares ahead. A model trained on queue position alone would correctly predict B is far more likely to fill than A within the horizon, even though they're the "same" order by every attribute except position in the queue — exactly the point of including it as a feature.

What this means in practice

Fill-probability estimates feed directly into the passive-versus-aggressive decision at the heart of every execution algorithm: when predicted fill probability at the current price is low, crossing the spread (or reposting closer to the market) becomes relatively more attractive despite its cost. Evaluation should always be done on a held-out set of orders the model never trained on, using proper classification metrics like log-loss or calibration curves rather than raw accuracy, since fills are often a minority class and a model that always predicts "no fill" can look deceptively accurate while being useless.