Retraining Cadence and Model Staleness

A model was trained a year ago and is still trading. Is that fine, or is it quietly degrading? Every model was fit on a specific window of history, and markets keep moving after that window closes — relationships between features and returns shift, volatility regimes change, and new instruments or regulations reshape the data. Never retraining risks running a model that has slowly drifted out of touch with the market it operates in. Retraining constantly, on the other hand, means refitting on tiny windows of new data, chasing noise, and introducing operational risk every single time a live model gets swapped. Getting the cadence right — and knowing when staleness has actually set in, rather than guessing on a fixed calendar — is its own discipline.

An analogy: how often to reprint a road map

Think of a paper road map of a fast-growing city. Reprint it too rarely — once a decade — and drivers get lost following streets that have since been rerouted or renamed. Reprint it every single day and you spend all your effort and expense on maps that are barely different from yesterday's, most of which contained no meaningful new information anyway; you're also more likely to introduce a printing error each time you rush a new edition out. The right update cadence depends on how fast the city is actually changing, not on an arbitrary fixed schedule — a booming construction zone might need monthly updates, while an established, stable neighborhood might go years unchanged.

Two ways to decide cadence

Fixed-schedule retraining: refit the model every $N$ weeks or months on a rolling or expanding window of recent data, regardless of whether performance has actually degraded. Simple to operate and audit, but blind — it retrains even when nothing has changed, and it may retrain too slowly if the market shifts faster than the schedule anticipates.

Trigger-based retraining: monitor a staleness signal directly — a rolling measure of live prediction accuracy, a distributional distance between recent live feature values and the training distribution (as in covariate shift), or a control-chart-style alarm on live P&L relative to backtest expectations — and retrain only when that signal crosses a pre-set threshold. This adapts to how fast the world is actually moving, but requires a reliable staleness metric, and a noisy metric can trigger unnecessary retrains or, worse, miss a real drift because the signal is too slow to react.

In practice, most production systems combine both: a fixed maximum interval as a backstop (never let a model go more than, say, 6 months without at least a refresh check) layered with trigger-based early retraining if a staleness signal fires sooner.

Worked example: comparing two cadences

A model's live accuracy is tracked weekly against its expected backtest accuracy of 58%. Under a fixed quarterly retraining schedule, the model runs from week 1 (58% accuracy) down to week 10 (52% accuracy, a real regime shift has occurred) before the next scheduled retrain at week 13 — three full weeks of degraded performance went unaddressed simply because the calendar hadn't come around yet. Under a trigger-based scheme with a threshold set at "3 consecutive weeks more than 4 points below expected accuracy," the system flags staleness at week 9 and retrains that week, cutting the degraded-performance window from three weeks to roughly zero. The trigger-based approach costs more operational complexity (building and maintaining the monitoring signal) but responds to the actual timing of the shift rather than an arbitrary calendar date.

What this means in practice

Retraining cadence should be set by how fast the specific model's relevant relationships actually move, not by an arbitrary convention borrowed from another team's strategy — a high-frequency microstructure model may need daily or weekly refreshes, while a slow-moving macro factor model may genuinely be fine for a year or more. Every retrain is itself a risk event that needs the same governance as any model change (backtesting the new fit, ideally shadow-testing it before promotion) — retraining too eagerly on a noisy staleness signal can introduce more instability than the staleness it was meant to fix.