Bad Tick and Outlier Detection
How trading systems tell a genuine, sharp price move apart from a data error that just looks like one, and the practical filters used to catch corrupted ticks before they poison a signal or trigger a bad trade.
Prerequisites: Data Quality Checks and Validation
A single garbled tick — a price reported at ten times its real value because of a decimal error, or a trade printed at a stale price from a different venue's feed — looks, for one instant, exactly like a genuine flash move. A strategy that reacts to price the moment it changes can't wait around to see if the move "sticks" before deciding whether to trust it; it has to make a real-time call about whether a suspicious tick is data or noise. Bad tick detection is the set of filters built to make that call automatically, fast enough to run inline before a strategy ever sees the tick.
The core difficulty is that legitimate markets really do move fast and far sometimes — a surprise news release can move a price 5% in a second, and a filter tuned to reject any large, sudden move will reject real information along with bad data. The practical filters used in production tend to combine several signals rather than relying on any single threshold: how far the price deviates from a short rolling average or from the best bid/ask on record, whether the same price appears on other venues at roughly the same time (a genuine move usually shows up nearly simultaneously across venues; a single bad tick usually doesn't), and whether the tick is immediately followed by a "correction" or reversion back to the prior level, which is a strong tell that the outlier print was spurious.
A common practical technique is to hold a suspicious tick in a short quarantine rather than immediately accepting or rejecting it — flag it, don't act on it yet, and see what the next tick or two says. If the price genuinely moved, the following ticks confirm it and the quarantined tick gets released; if it was a data error, the following ticks snap back and the outlier gets dropped or corrected. This costs a small amount of latency but avoids the two failure modes of a purely instantaneous filter: reacting to noise, or missing a real move because the filter was too conservative.
Worked example: a five-second quarantine catching a bad print
A stock trading steadily around $50.00 suddenly prints a trade at $5.00 — a plausible symptom of a fat-fingered order or a decimal error somewhere upstream. A naive strategy reacting instantly would see a 90% drop and could trigger a stop-loss or a buy-the-dip signal on pure noise. A quarantine filter instead flags the $5.00 print as suspicious (it's an order of magnitude off the rolling average and no other venue shows anything near it) and waits for the next few ticks. When the following trades come back in at $49.98 and $50.02, the filter concludes the $5.00 print was bad data, discards it, and never lets it reach the strategy's decision logic.
What this means in practice
Bad tick filters sit directly in the hot path of any strategy that reacts to individual prices, and getting the threshold wrong in either direction is costly — too aggressive and real moves get suppressed or delayed; too lax and garbage data drives real trades. Filters tend to be tuned per instrument, since a stock's normal volatility sets what counts as "suspiciously large," and re-tuned periodically as normal volatility itself shifts.
Bad tick detection has to distinguish a genuine sharp move from a data error using signals available in real time — deviation from a rolling reference, cross-venue confirmation, and whether the price reverts — because a filter based on move size alone will either reject real news or accept real garbage.
Related concepts
Practice in interviews
Further reading
- Kleppmann, Designing Data-Intensive Applications, ch. 4