Score-Based Models and Langevin Dynamics

Most generative models try to learn $p(x)$, the probability of seeing data point $x$, directly. That's harder than it sounds — $p(x)$ has to integrate to $1$ over all possible $x$, a global constraint that makes direct estimation awkward for complex, high-dimensional data. A score-based model sidesteps the problem by learning something that doesn't need to integrate to anything in particular: the score, $\nabla_x \log p(x)$ — the direction, at any point $x$, in which probability increases fastest. Knowing that direction everywhere turns out to be enough to generate samples from $p(x)$, without ever computing $p(x)$ itself.

The analogy: finding the top of a hill in the fog

Imagine standing somewhere on a hillside in thick fog, unable to see the whole landscape (you don't know $p(x)$), but able to feel which way is uphill from wherever you're standing (you know $\nabla_x \log p(x)$ at your current location). Repeatedly stepping uphill, using only that local feel, eventually gets you near a peak — a high-probability region — even though you never once saw the whole hill's shape. That's the entire strategy: learn the local "which way is more probable" direction everywhere, and let repeated small steps do the rest.

Langevin dynamics: uphill steps plus noise

Pure gradient ascent on $\log p(x)$ would march straight to the single highest peak and get stuck there, generating the same point every time. Langevin dynamics fixes this by adding a controlled amount of noise to every step:

$$x_{t+1} = x_t + \frac{\epsilon}{2} \nabla_x \log p(x_t) + \sqrt{\epsilon}\, z_t, \qquad z_t \sim \mathcal{N}(0, I)$$

In words: take a small step in the uphill (higher-probability) direction, sized by step size $\epsilon$, and then add a small random jolt scaled by $\sqrt{\epsilon}$. The uphill pull keeps you near high-probability regions; the noise stops you from collapsing onto one exact point, and, run for long enough with a properly decreasing $\epsilon$, the sequence of $x_t$'s ends up distributed according to $p(x)$ itself, not just clustered near its mode.

Worked example 1: one Langevin step by hand

Say $x_t = 3$, and the learned score at that point is $\nabla_x \log p(x_t) = -1.5$ (probability increases toward smaller $x$ here), step size $\epsilon = 0.1$, and a sampled noise $z_t = 0.4$. Then:

$$x_{t+1} = 3 + \frac{0.1}{2}(-1.5) + \sqrt{0.1}(0.4) = 3 - 0.075 + 0.126 = 3.051$$

The deterministic pull moved $x$ down by $0.075$ (toward higher probability), but the noise term pushed it up by more, landing slightly higher than $x_t$ overall — a reminder that any single step can move either way; it's the accumulated pull over many steps, with $\epsilon$ shrinking over time, that converges toward the true distribution's shape rather than one point.

Worked example 2: why the score doesn't need $p(x)$'s normalizing constant

Suppose the true (unnormalized) density is $\tilde{p}(x) = 5\,e^{-2(x-1)^2}$, so the true, normalized density is $p(x) = \tilde{p}(x)/Z$ for some constant $Z$ that makes it integrate to 1. The score is:

$$\nabla_x \log p(x) = \nabla_x \big[\log \tilde{p}(x) - \log Z\big] = \nabla_x \log \tilde{p}(x) = -4(x-1)$$

The constant $\log Z$ vanished under the gradient, since it doesn't depend on $x$. At $x=1$: score $=0$ (already at the peak). At $x=2$: score $=-4$ (strong pull back toward $1$). This is the whole reason score-based models are tractable where direct likelihood models struggle — the impossible-to-compute normalizing constant $Z$ simply never appears in the quantity being learned.

Each wiggly path above is exactly a Langevin trajectory with the deterministic drift term switched off — pure noise. Score-based sampling is this same random walk with a learned "pull toward higher probability" added back in at every step.

What this means in practice

Score-based models are the mathematical backbone of modern diffusion models: a diffusion model is, at its core, a network trained to estimate the score at every noise level, with sampling implemented as a discretized Langevin-like process run from pure noise down to a clean sample. Understanding the score/Langevin view explains why diffusion sampling needs many small steps rather than one big jump — each step is only a local, noisy uphill move, not a leap to the answer.