Classifier-Free Guidance

A diffusion model conditioned on a prompt $y$ — "a photo of a cat," or in a finance setting, "a stressed-regime return path" — will follow $y$ only loosely by default, because the training objective rewards matching the conditional data distribution $p(x\mid y)$, and that distribution still has plenty of room for outputs that only weakly resemble $y$. Earlier approaches used a separate trained classifier to push samples toward $y$ more forcefully, but that meant training and maintaining a second model, on noisy intermediate images the classifier had never seen clean. Classifier-free guidance gets the same forceful push using only the diffusion model itself, trained once.

The analogy: exaggerating the difference between two directions

Suppose you're giving driving directions and you know both "the route to the mall" and "the route to nowhere in particular" from the current junction. If someone follows "the mall route" only loosely, you can make the instruction more insistent not by hiring a separate guide, but by comparing the two routes and telling the driver to lean further in the direction the mall route diverges from the aimless one — exaggerating the difference between "going somewhere specific" and "going nowhere in particular" pushes them more decisively toward the mall.

The mechanics: train both, extrapolate at sampling time

During training, the same network predicts noise conditioned on $y$ some fraction of the time, and conditioned on nothing (a null label, $\varnothing$) the rest of the time — one network, two modes, achieved just by randomly dropping the conditioning during training. At sampling time, both predictions are computed for the same noisy point, and combined with a guidance scale $w > 1$:

$$\hat{\epsilon}_w = \epsilon_\theta(x_t, \varnothing) + w \big[\epsilon_\theta(x_t, y) - \epsilon_\theta(x_t, \varnothing)\big]$$

In words: start from the unconditional prediction, take the difference the conditioning makes, and add back more than 100% of that difference (since $w>1$). This exaggerates whatever the label $y$ was already nudging the prediction toward, at the cost of some diversity — the sample is pushed harder toward the specific mode $y$ implies, and correspondingly further from the variety of outputs the unconditional model would otherwise consider plausible.

Worked example 1: computing a guided prediction

Suppose at some diffusion step, the unconditional noise prediction is $\epsilon_\theta(x_t,\varnothing) = 0.20$ and the conditional one is $\epsilon_\theta(x_t,y) = 0.35$ (a single scalar for simplicity). With guidance scale $w=1$ (no guidance, plain conditional sampling): $\hat\epsilon = 0.20 + 1(0.35-0.20) = 0.35$ — just the conditional prediction. With $w=3$: $\hat\epsilon = 0.20 + 3(0.15) = 0.20+0.45=0.65$ — nearly double the conditional prediction's deviation from the unconditional one, pushing the denoising step much more aggressively toward whatever $y$ favors.

Worked example 2: the diversity trade-off, in numbers

Generate 100 samples at $w=1$: suppose 40 land in each of two plausible sub-styles consistent with $y$, and 20 are more generic/off-prompt. At $w=5$, the extrapolation collapses much of that spread: perhaps 85 land in the single dominant sub-style, 10 in the other, and only 5 generic — prompt-following goes up sharply (fewer off-prompt samples), but the model has effectively narrowed which of the two valid sub-styles it explores, which is the classic guidance trade-off: fidelity to $y$ rises, diversity within "things consistent with $y$" falls.

Picture the guidance scale $w$ as this curve's steepness parameter: small $w$ gives a gentle nudge toward the prompt, while large $w$ sharpens the response into an aggressive, low-diversity extrapolation — the same shape of trade-off between responsiveness and stability that shows up whenever you scale up a control signal.

What this means in practice

Every modern text-to-image and text-to-audio diffusion system uses some form of classifier-free guidance, and the same mechanism applies to a finance-conditioned generator — e.g. pushing synthetic scenario generation harder toward a specific stress label at the cost of exploring fewer distinct ways that stress could manifest. The guidance scale $w$ is a genuine dial practitioners tune per use case, not a fixed constant.