Determinism And Jitter Control

Ask a trading system's average latency and you'll get a reassuring number — say, 8 microseconds from receiving a market data update to sending an order out. But the average hides the number that actually matters for a strategy racing to cancel a stale quote or react to news: how bad is the worst case, and how often does it happen? A system that's usually 8 microseconds but occasionally spikes to 800 is far more dangerous than one that's a boringly consistent 15 microseconds every single time. That consistency is called determinism, and the variation around it is jitter.

Why the tail matters more than the average

In HFT, the moments that decide whether a strategy wins or loses money are exactly the moments when the system is under the most stress — a burst of market data during a fast move, a cancel racing to beat an incoming fill. Those are precisely the conditions most likely to trigger a slow outlier: a garbage collection pause, an operating system scheduling another process ahead of the trading thread, a cache miss, a queued network packet. A system's typical speed says almost nothing about the rare, high-stakes moment when speed matters most, which is why HFT engineering targets low jitter — a tight, predictable latency distribution — at least as hard as a low average.

Worked example: two systems with the same average

System A sends an order in 8 microseconds on average, but its distribution has a long tail: 1 in 10,000 updates takes over 500 microseconds because of an occasional garbage-collection pause. System B is slower on average, 15 microseconds, but extremely consistent: 99.99% of updates fall between 14 and 17 microseconds. In calm markets, System A looks better — it's faster nearly all the time. But in the rare instant that decides whether the desk cancels a stale quote before it's picked off, System A has roughly a 1-in-10,000 chance of being catastrophically slow at exactly the wrong moment, while System B's worst case barely differs from its typical case. Where the tail event is where the money is won or lost, System B's determinism beats System A's better average.

What this means in practice

Engineering for determinism means removing sources of unpredictable delay rather than just optimizing the common case: pinning threads to dedicated CPU cores so the operating system never preempts them, avoiding memory allocation (and the garbage collection it can trigger) on the critical path entirely, using kernel-bypass networking to skip variable operating-system network stack delays, and measuring performance in percentiles — the 99.9th or 99.99th percentile latency — rather than trusting an average that a rare tail event barely moves.