Catastrophic Forgetting and Continual Learning

A neural network trained on task A, then fine-tuned on task B without any special precautions, will often lose most of its ability on task A — not gradually, but sharply, sometimes after only a modest amount of task-B training. This is catastrophic forgetting, and it isn't a bug in any particular implementation; it's close to the default behavior of gradient descent applied to a shared set of weights, because nothing in the ordinary training objective for task B has any reason to preserve whatever those weights previously encoded for task A. Continual learning is the broader problem of training on a sequence of tasks over time without this collapse — building a model that accumulates skills rather than trading old ones for new ones.

The analogy: painting over a canvas instead of using a new one

A painter working on a single, reused canvas, painting one scene, then wiping it and painting a second scene, ends up with only the second scene — the first is gone unless the painter deliberately protected part of the canvas. That is what happens to a network's weights during naive sequential training: task B's gradients push weights in whatever direction reduces task B's loss, with zero regard for whether that direction also destroys the pattern of weights that made task A work. Nothing about ordinary gradient descent "remembers" that some weights mattered for a different purpose — unless you build that protection in explicitly.

Building in protection

One influential approach, Elastic Weight Consolidation (EWC), identifies which weights mattered most for task A (using the Fisher information, a measure of how sensitive task A's loss is to each weight) and adds a penalty during task B training that discourages moving those specific weights far from their task-A values:

$$L_B'(\theta) = L_B(\theta) + \frac{\lambda}{2}\sum_i F_i (\theta_i - \theta_{A,i}^*)^2$$

In words: the new loss for task B ($L_B'$) is the ordinary task-B loss ($L_B$) plus a penalty term that, for every weight $i$, punishes moving away from its task-A optimal value ($\theta_{A,i}^*$) in proportion to how important that weight was for task A (its Fisher information $F_i$) — weights that barely mattered for task A get little protection and are free to change for task B; weights that mattered a lot get anchored in place.

Worked example 1: forgetting without protection

A network trained on task A reaches 95% accuracy on task A. After 500 steps of naive fine-tuning on task B (no protection), task A accuracy might drop to 40% — worse than many simple baselines — even though task B accuracy has climbed to 90%. The network didn't gradually trade a little of task A for task B; it lost the majority of task A's performance while gaining most of task B's, because gradient descent on task B alone had no incentive to preserve anything about task A.

Worked example 2: EWC's protective anchor

Same setup, but with EWC and $\lambda = 1000$, a weight $\theta_i$ that had high Fisher information $F_i = 50$ for task A and originally sat at $\theta_{A,i}^* = 0.8$: if training on task B alone would push it to $\theta_i = 0.3$ (a shift of $0.5$), the EWC penalty adds a cost of $\frac{1000}{2}\times 50 \times (0.3-0.8)^2 = 500 \times 50 \times 0.25 = 6{,}250$ to the loss for that single weight's shift — a large enough penalty that gradient descent on the combined loss will pull that weight back much closer to $0.8$ than task B's loss alone would have. A different weight with low Fisher information, say $F_j=0.1$, gets a penalty of only $500\times0.1\times0.25=12.5$ for the same-sized shift — negligible, so it moves freely.

What this means in practice

Catastrophic forgetting matters anywhere a model keeps learning after deployment — a recommendation system adapting to new behavior, a robot learning new skills without losing old ones, a language model fine-tuned repeatedly on new domains. Mitigations beyond EWC-style regularization include replaying old-task data alongside new-task data, or allocating separate parameters per task — each trades memory, compute, or flexibility for protection, and none eliminates the problem entirely over arbitrarily long task sequences.

Practice

1. A weight has Fisher information $F=20$ for task A, originally at $0.5$; task B's loss alone would move it to $0.1$. With $\lambda=500$, what is the EWC penalty for that shift?
2. Why doesn't ordinary gradient descent on task B alone have any built-in reason to preserve task A's performance?
3. Name one continual-learning mitigation besides EWC-style weight penalties.