The Wasserstein Distance

Total variation distance can be blind to something that obviously matters: how far apart two distributions are, not just whether they overlap. Two point masses at 0 and 1 have total variation distance 1 — the maximum possible. So do two point masses at 0 and 1,000,000. Total variation distance can't tell those apart, but any sane notion of "distance" between distributions should say the second pair is much further apart. The Wasserstein distance fixes exactly this.

The analogy: moving piles of dirt

Imagine distribution $P$ as a pile of dirt shaped one way and $Q$ as a target shape you need to reshape it into. You can move dirt around, but moving a shovelful a long distance costs more than moving it a short distance — cost is (amount of dirt moved) × (distance moved). The Wasserstein distance (also called earth mover's distance) is the minimum total cost of any plan that reshapes $P$ into $Q$. Unlike total variation distance, which only asks "do the piles overlap," Wasserstein directly charges for physical distance — which is exactly why it distinguishes "1 unit away" from "a million units away."

Writing it down

For distributions $P, Q$ on the real line, the (order-1) Wasserstein distance is

$$W_1(P,Q) = \inf_{\gamma \in \Gamma(P,Q)} \int |x - y| \, d\gamma(x,y),$$

where $\Gamma(P,Q)$ is the set of all couplings of $P$ and $Q$ (joint distributions with those marginals — the same idea used for total variation distance). In words: among every possible way of jointly pairing draws from $P$ with draws from $Q$, find the pairing that minimizes the average distance moved. On the real line this has a strikingly simple closed form using the cumulative distribution functions $F_P, F_Q$:

$$W_1(P,Q) = \int_{-\infty}^{\infty} |F_P(x) - F_Q(x)| \, dx.$$

In words: stack up the area between the two CDF curves — no optimization needed on the line, just integrate the gap between how much probability each distribution has accumulated by each point.

Picture two CDF-like rising curves instead of this one: the vertical gap between two such curves at every point, summed up (the area between them), is exactly $W_1$ — a direct visual of "how much probability mass sits in the wrong place, and how far it has to travel."

Worked example 1: two point masses

$P$ is a point mass at 0, $Q$ a point mass at 5. There's only one possible coupling (both are deterministic), so all the "mass" travels a distance of 5: $W_1(P,Q) = 5$. Compare total variation distance for the same pair: $d_{TV} = 1$, the maximum possible, exactly as it would be if $Q$ were a point mass at 5 or at 5,000,000 — TV distance can't see the difference; Wasserstein reports 5 versus 5,000,000, which is the answer that actually matches intuition.

Worked example 2: two uniform distributions

$P = \text{Uniform}(0,1)$, $Q = \text{Uniform}(2,3)$. Their CDFs never overlap in support, so $F_P(x) - F_Q(x)$ is nonzero across a range of width roughly 3. Concretely: for $x\in[0,1]$, $F_P(x)=x, F_Q(x)=0$, contributing $\int_0^1 x\,dx = 0.5$. For $x\in[1,2]$, $F_P=1, F_Q=0$, contributing $\int_1^2 1\,dx = 1$. For $x\in[2,3]$, $F_P=1, F_Q=x-2$, contributing $\int_2^3 (1-(x-2))\,dx = 0.5$. Total: $0.5+1+0.5 = 2$ — matching the intuitive answer that shifting a unit-width block from $[0,1]$ to $[2,3]$ moves each point a distance of 2. Total variation distance, again, would simply report 1 (complete disjointness) whether the shift were by 2 or by 200.

Drag the mean control here and watch two curves separate — the further apart the peaks sit, the more area sits between the two CDFs, and the larger the Wasserstein distance grows in direct proportion, unlike total variation distance which would stay pinned at its maximum the moment the curves stop overlapping at all.

What this means in practice

• Comparing return distributions across regimes. Wasserstein distance between a strategy's pre-crisis and post-crisis P&L distribution quantifies not just "did the distribution change" but "by how much, in the actual units of P&L" — directly interpretable, unlike a KL-divergence or TV-distance number.
• Generative models. Wasserstein GANs train generators by minimizing exactly this distance between real and synthetic data distributions, precisely because it stays well-behaved (gives a useful gradient) even when the two distributions have little or no overlap — a case where TV distance and KL divergence go flat or infinite.
• Stress-testing scenario generation. Measuring how far a stress scenario's distribution sits from history, in real economic units, is a natural Wasserstein application.