Teacher Forcing and Exposure Bias

A model that predicts a sequence one step at a time — tomorrow's price move, the next word in a sentence, the next token in a forecast — has to decide what to feed itself as "the recent past" before predicting the next step. During training you know the true past exactly. But at test time there is no true past to hand it: the model has to use its own earlier predictions, and those are never perfect. This mismatch between what the model sees while learning and what it sees while working is the whole problem.

The analogy: rehearsing with a script versus performing live

Imagine training an actor by having them read each line off a perfect script, never having to remember or guess the line before. They nail every line in rehearsal. But on stage, there is no script feeding them the previous line — they have to remember what they themselves just said, slip included. If they never practiced recovering from a flubbed line, one small stumble can throw off everything after it, because they never learned what to do once the "true" script and their own memory diverge.

The mechanics: teacher forcing during training

Teacher forcing means that at each training step $t$, the model predicts the next output $\hat{y}_t$ using the true previous output $y_{t-1}$ as input, not its own prediction $\hat{y}_{t-1}$:

$$\hat{y}_t = f(y_{t-1}, y_{t-2}, \dots, h_t)$$

Here $f$ is the model, $h_t$ is its hidden state, and every $y$ on the right-hand side is the ground truth from the training data, not something the model generated. In words: the "teacher" always hands the model the correct answer for the previous step, regardless of what the model would have guessed. This makes training fast and numerically stable — errors from one step can't compound into the next, so gradients stay well-behaved and the loss at each position can be computed independently.

At inference time, though, there is no ground truth left to hand over. The model must condition on its own prior output:

$$\hat{y}_t = f(\hat{y}_{t-1}, \hat{y}_{t-2}, \dots, h_t)$$

That single substitution — true $y$ swapped for predicted $\hat{y}$ — is the entire source of exposure bias: the model was never exposed, during training, to sequences built from its own imperfect predictions, so it has no learned behavior for correcting a self-generated error before it snowballs.

Worked example 1: a toy price-direction sequence

Suppose a model forecasts a 5-day sequence of up/down moves, true sequence Up, Up, Down, Up, Down. Training with teacher forcing, at day 3 the model is told the true day-2 value (Up) regardless of what it predicted for day 2. It learns "given true history …Up, Up…, predict Down" perfectly. But suppose at test time the model mispredicts day 1 as Down. At day 2 it now conditions on its own wrong "Down," a history it may never have seen in training, with no learned rule for it — and the error can compound through days 3, 4, and 5.

Worked example 2: measuring the drift

Say a demand-forecasting model has a 5% per-step chance of a self-generated error large enough to push the next input outside the range it saw in training. Over a 20-step forecast, the chance of staying entirely within familiar territory the whole way is roughly $(0.95)^{20} \approx 0.36$ — under 40%. So for most 20-step forecasts, the model will, at some point, be forced to extrapolate from a self-generated input it never practiced recovering from.

Watch how a single early deviation in a generated path can pull the whole trajectory away from where a "clean" path would sit — that is the geometric shape of exposure bias compounding over steps.

What this means in practice

Production sequence models often mix in some self-generated inputs during training — most simply, Scheduled Sampling, which gradually replaces true previous outputs with the model's own predictions as training progresses, giving practice recovering from mistakes before doing so unsupervised. For forecasting, this interacts directly with the choice between recursive and direct multi-horizon strategies, since a purely recursive model is maximally exposed to this compounding.

Practice

1. Explain why teacher forcing makes gradients more stable to compute than training on self-generated sequences.
2. If a model's per-step self-generated error rate is 2%, what is the probability it stays "in-distribution" for a full 50-step generation?
3. Name one training-time change (other than scheduled sampling) that would reduce exposure bias.