Venue Selection With Contextual Bandits

A smart order router has to decide, for every child order it sends out, which venue to send it to among a dozen or more exchanges and dark pools, each with different fee structures, fill rates, and adverse-selection profiles. This decision repeats thousands of times a day and the "right" venue genuinely shifts — a pool that was cheap and clean last month can quietly turn toxic as its participant mix changes. A contextual bandit is the natural framework for learning this kind of repeated choose-one-of-several-options decision directly from outcomes.

Why "bandit," and why "contextual"

A bandit problem is named after a slot machine ("one-armed bandit"): you have several arms to pull (here, venues to route to), each with an unknown payoff distribution, and you learn about each arm's payoff only by pulling it. The core tension is exploration versus exploitation — routing everything to the venue that's looked best so far (exploitation) risks missing that another venue has quietly become better, while constantly trying new venues (exploration) costs money on the ones that turn out worse. "Contextual" means the best arm isn't fixed — it depends on the context at the time of the decision: order size, time of day, current volatility, the stock itself. A venue that's excellent for small passive orders in a quiet stock might be a poor choice for a large aggressive order in a fast-moving one.

Setting it up

Each routing decision is one bandit "pull": the context is the feature vector describing the order and current conditions, the arm chosen is the venue, and the reward is a measured outcome like effective spread paid, fill rate, or post-fill markout (a proxy for adverse selection at that venue). A policy — often a simple model like a contextual version of Thompson sampling or an epsilon-greedy scheme layered on a reward-prediction model — is updated continuously as new fills arrive, so a venue whose recent fills have started showing worse markouts gets deprioritized without needing a full model retrain from scratch.

Worked example: two venues, converging evidence

A router is choosing between venue A and venue B for small passive orders in a mid-cap name. Early on, both look similar: A shows an average 2bp cost advantage per fill on the first 50 routed orders, well within noise given the small sample. Because the policy still allocates a meaningful exploration fraction (say, 15% of orders) to B even while favoring A, it accumulates 200 more fills on each venue over the next week. That larger sample reveals A's average effective spread is consistently about 1.5bp better than B's, with markouts also modestly better — evidence solid enough to shift the policy toward routing 85% of similar orders to A going forward, while still sending a small fraction to B in case conditions there improve or A's advantage erodes.

What this means in practice

Framing venue selection as a contextual bandit rather than a static ranking is what lets a router adapt as venue quality drifts over time — mechanically, through changing fee schedules and matching logic, or behaviorally, as the mix of participants on a venue changes. The exploration fraction is a real cost, not a free lunch: it deliberately routes some orders to venues believed to be worse, which is the price of keeping the router honest about venues that may have improved, and needs to be tuned to the actual cost of being wrong versus the cost of never finding out.