Reconciliation Tolerances Between Implementations

Two systems computing "the same" number — a backtest P&L and a live P&L, or a research signal and its production port — will almost never agree exactly. Floating-point rounding, slightly different data timestamps, or a marginally different fill assumption all introduce tiny discrepancies. A reconciliation tolerance is the pre-agreed threshold below which a difference is shrugged off as noise, and above which it triggers an investigation. Without one, a team either chases phantom bugs forever or, worse, waves away a real one because "it's close enough."

Setting the tolerance is a judgment call, not a formula: it should be small enough to catch genuine logic errors but large enough to absorb known, harmless sources of drift. A common approach is a relative tolerance (say, 1 basis point of notional) plus a small absolute floor, since a pure percentage breaks down near zero. The tolerance is usually tightened over time as a system matures — wide during initial build-out, narrow once both sides are trusted.

For example, a research backtest might report a day's P&L as $10,042 while the live system reports $10,058 — a $16 difference, or about 0.16%. If the agreed tolerance is 0.25% of daily P&L, this passes without action. If the next day the gap widens to 3%, that's a signal something changed: a data feed, a fill model, or an actual bug, and it needs tracing before more capital rides on the mismatched number.