Backpropagation Through Time

A recurrent network processes a sequence one step at a time, carrying a hidden state forward and using the same set of weights at every single step — that weight sharing is what lets it handle sequences of any length with a fixed number of parameters. But it creates a puzzle for training: if a mistake shows up at timestep 50, and the weights that produced it were also used at timesteps 1 through 49, how much of the blame belongs to each use? Ordinary backpropagation was built for a single forward pass through a fixed stack of distinct layers. A recurrent network run for 50 steps behaves, for gradient purposes, like a 50-layer network where every layer happens to share identical weights — and the gradient has to account for that sharing correctly, not just once, but by summing the weight's influence across every single timestep it touched.

The analogy: one employee blamed across fifty shifts

Imagine one employee works the same register, using the exact same habits, for fifty shifts in a row, and at the end of the month the store finds a total cash shortfall. To figure out how much to adjust that employee's training, you can't just look at shift 50 — the habits that caused shift 50's shortfall are the same habits used in shifts 1 through 49, so you need to trace the shortfall's cause back through every shift those habits touched and add up the total damage attributable to that one set of habits. That's backpropagation through time: unroll the recurrent network into "one copy per timestep" that all secretly share the same weights, run ordinary backpropagation on the unrolled chain, and then sum the gradient contributions from every timestep back onto the single shared weight, because that one weight was responsible for the error at every step it participated in.

The maths

A vanilla recurrent network updates its hidden state $h_t$ from the previous state and the current input:

$$h_t = \tanh(W_h h_{t-1} + W_x x_t + b)$$

In words: the new hidden state blends the old hidden state $h_{t-1}$ (via weight matrix $W_h$) with the new input $x_t$ (via $W_x$), squashed through $\tanh$. Crucially, $W_h$, $W_x$, and $b$ are the same matrices and vector at every timestep $t$ — there is only one copy of each, reused $T$ times for a length-$T$ sequence.

To train, you need $\partial \mathcal{L}/\partial W_h$, where $\mathcal{L}$ is the total loss summed over all timesteps. Because $W_h$ was used at every step, the chain rule requires summing its influence across all of them:

$$\frac{\partial \mathcal{L}}{\partial W_h} = \sum_{t=1}^{T} \frac{\partial \mathcal{L}_t}{\partial W_h}$$

In words: work out how much timestep $t$'s loss $\mathcal{L}_t$ would change if you nudged $W_h$, do that separately for every timestep, then add all $T$ contributions together — because $W_h$ is literally one shared variable, its total gradient is the sum of every place it was used.

Each individual term $\partial \mathcal{L}_t / \partial W_h$ itself requires chaining back through every hidden state between $W_h$'s use at some earlier step and the loss at step $t$:

$$\frac{\partial \mathcal{L}_t}{\partial W_h} = \sum_{k=1}^{t} \frac{\partial \mathcal{L}_t}{\partial h_t} \left( \prod_{j=k+1}^{t} \frac{\partial h_j}{\partial h_{j-1}} \right) \frac{\partial h_k}{\partial W_h}$$

In words: to find how the loss at step $t$ depends on the weight's use at some earlier step $k$, you multiply together the local derivative $\partial h_j / \partial h_{j-1}$ at every single step in between — a long product of Jacobian matrices, one per timestep spanned. This product is the crux of the whole method, and also its Achilles' heel: each factor is typically well below 1 in magnitude (because $\tanh$'s derivative maxes out at 1 and is usually smaller), so a long product of them shrinks toward zero as $t - k$ grows. That is the vanishing gradient problem, and it means the network effectively stops learning from anything that happened more than a handful of steps in the past.

Worked example 1: the vanishing product, with real numbers

Suppose the local Jacobian $\partial h_j/\partial h_{j-1}$ has magnitude roughly $0.6$ at every step — plausible once $\tanh$ has saturated somewhat and $W_h$ isn't unusually large. To find how much the loss at $t=10$ depends on the hidden state at $k=2$, the chain requires the product across 8 intervening steps:

$$\prod_{j=3}^{10} \frac{\partial h_j}{\partial h_{j-1}} \approx 0.6^{8} = 0.0168$$

Only 1.7% of the gradient signal from step 10's loss survives the trip back to step 2. Push the gap to 20 steps instead of 8:

$$0.6^{20} \approx 0.0000366$$

That's a signal reduced to 0.0037% of its original size — for all practical purposes zero. Any dependency the network needed to learn that spans 20 or more timesteps is invisible to gradient descent, not because the pattern isn't there in the data, but because the training signal that would teach it has decayed to numerical noise before it arrives.

Worked example 2: truncated BPTT and its cost

Training on a 10,000-step sequence by unrolling the full thing is both slow and where the vanishing product is at its worst. Truncated BPTT instead breaks the sequence into chunks — say chunks of 20 — and only backpropagates within each chunk, carrying the hidden state forward across chunk boundaries but not the gradient.

Consider a dependency in the data between an event at absolute position 1,005 and an outcome at position 1,015 (10 steps apart, within one chunk boundary): truncated BPTT with chunk length 20 can, in principle, still learn it, since both fall in the same or adjacent windows.

But a dependency between position 1,000 and position 1,050 (50 steps apart) spans more than two chunk boundaries. Even though the hidden state carries some trace of position 1,000 forward mechanically, no gradient ever flows back across the chunk boundary to explicitly teach the network "the outcome at 1,050 depends on what happened at 1,000" — that credit assignment is structurally impossible with chunk length 20 spanning only local windows. Truncation trades the ability to learn long dependencies for tractable compute; the trade-off is explicit and the chunk length is the parameter that sets how much you're giving up.

Watch how a step that's too small crawls without visibly progressing — that's the practical symptom of vanishing gradients in a recurrent network: the update to $W_h$ from a distant timestep is so small it's nearly indistinguishable from no update at all, even though the optimizer is technically still running.

What this means in practice

Any time a plain RNN seems to "forget" something from many steps back — an order-book imbalance from an hour ago that should still matter, a macro release from earlier in the session — the vanishing gradient through BPTT is very often the actual mechanism, not a data problem. This is precisely the failure mode that motivated gated architectures like the LSTM, which add a separate additive pathway for the cell state specifically so its local Jacobian along that path stays close to 1 rather than shrinking every step. When training any recurrent architecture, gradient clipping (capping the gradient's norm) guards against the opposite failure — exploding gradients, where the same product grows rather than shrinks — and is close to mandatory in practice.