Bad Tick Filtering

Exchange feeds occasionally print prices that are simply wrong. A trader fat-fingers an order and briefly trades a stock at ten times its price. An exchange's matching engine hiccups and reports a trade at zero. A vendor's feed handler drops a decimal point and turns $150.25 into $15,025. Every one of these gets recorded in the raw tape exactly like a real trade, with a real timestamp and a real (if wrong) price, and unless something catches it, that single bad tick will corrupt every downstream calculation that touches it — a return series with an impossible 900% one-tick spike, a moving average yanked off course, a volatility estimate inflated by an event that never actually happened in the market.

How filters actually work

Most practical bad-tick filters compare each new price against a short local reference — a rolling median or a recent trade price — and flag anything that deviates by more than some threshold, often expressed as a number of standard deviations of recent price changes or as a hard percentage move that's implausible for the instrument and timeframe (a stock rarely moves 5% in a single millisecond). A flagged tick is either dropped, replaced with the last good price, or held for a brief window to see if a following tick confirms the move was real. That confirmation window matters: a genuine, fast-moving news event can produce a large real price jump that looks exactly like a bad tick at first glance, and a filter that's too aggressive will delete real market moves along with genuine errors — which is its own kind of data corruption, arguably a worse one because it silently understates real volatility and risk.

A concrete case

An equity feed shows a stock trading at $52.10 for an hour, then a single print at $5.21, then back to $52.14 a millisecond later. A naive filter checking only "did the price change a lot" might miss this if it only compares consecutive ticks, since $52.10 to $5.21 and $5.21 back to $52.14 both look like enormous, isolated moves rather than one clear round-trip error. A filter that instead compares each tick to a short rolling window catches it immediately: $5.21 is wildly inconsistent with the surrounding cluster of $52-something prints, while $52.10 and $52.14 are consistent with each other and with everything nearby.

What this means in practice

Bad tick filtering has to run before, or as part of, any bar construction, signal calculation, or risk metric — a single unfiltered bad tick upstream can silently poison everything downstream. The design tension is unavoidable: too loose and errors leak through; too tight and real volatility gets erased. Most production pipelines log every filtered tick rather than deleting it outright, so a human can periodically audit whether the filter is making the right calls.