Learning Cancel-and-Replace Policies

A resting limit order isn't a "set and forget" instruction. As the market moves, an order that started at a good price can become stale — sitting behind a queue that will never clear, or perched at a price the market has drifted away from — and the algorithm must decide whether to leave it, cancel and repost closer to the market, or cancel and cross the spread entirely. The traditional approach hard-codes this as a rule: "cancel if not filled within 3 seconds" or "cancel if the mid moves more than one tick away." Learning the cancel-and-replace policy from data replaces that fixed rule with a decision that adapts to the actual state of the book.

Why a fixed timer is the wrong default

A flat timer treats every order the same regardless of context. An order sitting at the best bid with a short queue and heavy incoming buy pressure is likely to fill soon and shouldn't be canceled just because three seconds passed; an order that's fallen three ticks behind the market after a sudden move should probably be canceled immediately, not after some arbitrary wait. A learned policy conditions the cancel decision on the same features that drive fill probability — queue position, distance from mid, recent trade flow, order-book imbalance — so the wait time is effectively different for every order, shaped by what's actually happening around it rather than a clock.

Framing it as a decision problem

At each moment a resting order is live, there are three actions: hold, cancel-and-repost (usually closer to the touch), or cancel-and-cross (pay the spread to guarantee the fill now). The policy is learned by comparing, across historical episodes, the realized cost of each action given the state at that moment — this can be done with a supervised model that predicts the expected cost of holding a few seconds longer versus reposting, or with reinforcement learning that directly optimizes cumulative cost over the life of the parent order. Either way, the label isn't "did it fill" alone; it's the total cost outcome, since a fill obtained by crossing the spread is not free just because it happened.

Worked example: two orders, one obvious call

Order A rests at the best bid with 300 shares ahead of it in the queue, and the last ten seconds of tape show buy-side trades outpacing sell-side trades three to one — book conditions historically associated with an 80% chance of filling within the next 5 seconds. Order B rests at the second price level, two ticks behind a fast-moving market that has already drifted one tick away in the last two seconds, with a historical fill probability under 10% in similar states. A learned policy holds order A (canceling now would just forfeit a high-probability fill and force paying the spread) and cancels-and-reposts order B immediately (waiting only lets it drift further behind at zero benefit). A fixed 3-second timer would treat both identically and hold both for the same duration, needlessly delaying the repost on B.

What this means in practice

Learned cancel-and-replace policies matter most in fast, liquid names where the book turns over every few seconds and a fixed rule is either too slow to react to genuine drift or too jumpy in response to noise. They're typically layered on top of, not instead of, a fill-prediction model, since the cancel decision is really just "is the current fill probability high enough to justify the risk of waiting versus the cost of crossing." The main practical risk is training on historical data generated by the desk's own old (fixed-timer) policy, which never observed what would have happened under the actions the new policy actually wants to take — a gap that counterfactual evaluation techniques are needed to close.