Live Versus Backtest Reconciliation

A strategy backtested to a Sharpe of 1.6 goes live. After four months it's running at a live Sharpe of 0.4. The obvious read is "the edge decayed" — markets changed, other participants found the same signal, and the strategy is simply weaker now than it was in history. The actual cause, found only by comparing individual live fills against what the backtest assumed those same trades would have cost: the backtest modelled a fixed 3 bps of slippage per trade, and live execution, on a strategy that trades a genuinely illiquid basket, was averaging 11 bps. The edge hadn't decayed at all. The simulation had been wrong about costs from day one, and nobody could tell the two explanations apart without reconciling live fills against backtest assumptions trade by trade.

What reconciliation actually checks

Live-versus-backtest reconciliation takes each live trade and compares it against what the backtest, run over the exact same period and signals, predicted for that trade — fill price, cost, timing, size — rather than comparing only the two aggregate P&L curves. Comparing aggregate curves alone confounds every possible source of divergence into one number; comparing trade by trade isolates which assumption broke, because a slippage-model error, a latency difference, and genuine signal decay each leave a different fingerprint in the trade-level data.

Worked example

Diagnostic	Backtest assumption	Live observed	Signature
Slippage per trade	3 bps	11 bps	Cost gap is constant across the sample period — not growing over time
Signal strength (pre-cost return per trade)	18 bps	17 bps	Nearly identical — the signal itself is intact
Fill rate on limit orders	92%	90%	Close enough to rule out an execution-access problem

The pre-cost signal strength barely moved; the slippage assumption was off by nearly 4x and stayed off consistently rather than drifting, which is the signature of a mis-specified cost model, not decaying alpha. Fixing the backtest's slippage assumption to match reality (and testing the strategy again under the corrected model) would have caught this before a single dollar traded — the live Sharpe wasn't a surprise about the market, it was a surprise about the backtest.

The other direction: when live outperforms the backtest

Reconciliation isn't only for disappointing strategies. A strategy that runs better live than its backtest predicted deserves the same scrutiny, because the usual cause isn't extra skill showing up for the first time — it's a backtest that was too conservative or, more worryingly, a live process that's quietly taking on risk the backtest never modelled. A common version: a backtest that caps position size strictly at the stated limit, while the live execution system rounds order sizes up to the nearest lot and occasionally exceeds the intended cap by a small margin on illiquid names, adding return and risk that the backtest's cleaner position logic never captured. Trade-by-trade reconciliation catches this the same way it catches underperformance — by comparing what each individual trade was supposed to look like against what it actually was, rather than accepting a favourable aggregate number without asking why.

Reconciliation depends on the original backtest being reproducible enough to re-run against live-period data for comparison, which is why it only works reliably alongside the discipline in Reproducible Research Pipelines. See Slippage Models for the cost-modelling half of this gap and Monitoring Signal Decay in Production for how to separate genuine decay from a modelling error once the cost side has been ruled out.