Market Data Feed Handlers

An exchange's raw market-data feed is a firehose: millions of small, binary UDP multicast packets per second, each in the exchange's own proprietary format, arriving with gaps that need detecting and filling, and no guarantee they show up in order. Nothing further up a trading system — the order book, the strategy logic, the risk checks — wants to deal with any of that directly. The feed handler is the piece of software whose entire job is to absorb this raw mess and turn it into something clean: a normalized, gap-free, correctly sequenced stream of events that every downstream consumer can rely on without knowing anything about the exchange's specific wire protocol.

Concretely, a feed handler does three things. First, decoding: parse the exchange's binary wire format into structured messages (a new order, a cancel, a trade), which for a busy exchange must happen at extremely high throughput, often justifying the binary-format investment discussed elsewhere. Second, sequencing and gap recovery: since UDP provides no ordering or delivery guarantee, the handler tracks sequence numbers per message, detects any gap, and requests a retransmission or waits for a periodic snapshot to fill it — all without stalling the live stream of everything that isn't affected by that gap. Third, normalization: translating each exchange's own idiosyncratic message format and semantics into a common internal representation, so a strategy or order book built once can consume data from many different exchanges without being rewritten for each one's specific quirks.

Worked example: normalization saving downstream work

A firm trades on two exchanges: Exchange A represents a trade cancellation as a distinct message type with its own fields, while Exchange B represents the same event as a "modify" message with a quantity set to zero. Without normalization, the order-book logic and every strategy consuming this data would need separate, exchange-specific branches to recognize both representations as "this trade no longer counts" — doubling the surface area for bugs every time a third exchange with its own quirks is added. A feed handler that normalizes both into a single internal Cancel event means the order-book code is written once, against one clean event type, and adding a third exchange means writing one new decoder inside the feed handler rather than modifying every downstream consumer that touches market data.

What this means in practice

Feed handlers typically run as their own dedicated, often core-pinned processes on the most latency-critical path, since any delay or bug here propagates to everything downstream. Because gap detection and retransmission logic is genuinely tricky to get right — and getting it wrong silently produces a subtly corrupted order book rather than an obvious crash — feed handlers are usually among the most heavily tested and monitored components of a trading system, with dedicated alerting on sequence gaps, stale feeds, and reconnection events.