Convergence in Distribution

The central limit theorem says a sample mean, suitably rescaled, "converges to a normal distribution." But converges how, exactly? It is tempting to picture individual numbers homing in on individual target values, the way $1/n \to 0$. That picture is wrong here, and getting it wrong causes real confusion about what the CLT actually promises. What actually settles down is the shape of the whole distribution — the histogram, not any individual draw.

The analogy: a histogram that stops changing shape

Imagine rolling a die $n$ times and plotting a histogram of the average of those rolls, then repeating the whole experiment thousands of times to build up that histogram. For small $n$ the histogram is lumpy and asymmetric. As $n$ grows, the histogram smooths into a bell shape and that bell shape stabilises — draw the histogram for $n=1000$ and $n=2000$ and they look nearly identical in shape (once you rescale for the shrinking spread). Individual averages are still bouncing around, never sitting still — but the overall picture, the histogram's shape, has settled. That settling of the shape, not of any single number, is convergence in distribution.

Writing it down

A sequence of random variables $X_n$ converges in distribution to $X$, written $X_n \xrightarrow{d} X$, if

$$\lim_{n\to\infty} F_{X_n}(x) = F_X(x) \quad \text{for every } x \text{ where } F_X \text{ is continuous} .$$

In words: the CDFs of the sequence get closer and closer to the CDF of the limit, at every ordinary point — the shape of the distribution stabilises. Crucially, this says nothing about individual outcomes $X_n(\omega)$ for a specific random draw $\omega$ getting close to $X(\omega)$; it is entirely a statement about the two curves $F_{X_n}$ and $F_X$ matching up. This is the weakest of the standard modes of convergence — weaker than convergence in probability, which is itself weaker than almost-sure convergence — precisely because it only constrains aggregate behaviour, not individual paths.

Worked example 1: sample proportions converging to a normal shape

Flip a fair coin $n$ times and look at $Z_n = \dfrac{\hat p_n - 0.5}{\sqrt{0.25/n}}$, the standardised sample proportion of heads. For $n=10$: possible values of $\hat p_n$ are multiples of $0.1$, so the distribution of $Z_n$ is lumpy — e.g. $P(\hat p_{10}=0.5) = \binom{10}{5}(0.5)^{10} \approx 0.246$, a sizeable spike at the centre. For $n=1000$: the same standardised quantity has $P(0.49 \leq \hat p_{1000} \leq 0.51)$ computable via the normal approximation as $P(-0.63 \leq Z \leq 0.63) \approx 0.472$, and by this point the whole curve of $Z_n$'s distribution — not just this one probability — closely tracks the standard normal CDF to several decimal places at every $x$. This is the CLT in action: $Z_n \xrightarrow{d} N(0,1)$, and the shape has essentially converged well before $n=1000$, even though $\hat p_n$ itself is still a genuinely random, bouncing quantity at every $n$.

Worked example 2: why individual values don't converge, only the shape

Let $X_n = (-1)^n Z$ for a fixed standard normal $Z$ (so $X_n$ literally flips sign each step, alternating between $Z$ and $-Z$ forever). Does $X_n \xrightarrow{d} N(0,1)$? Check the definition: $F_{X_n}(x) = P((-1)^nZ \leq x)$. When $n$ is even this is $P(Z\leq x) = \Phi(x)$; when $n$ is odd it's $P(-Z \leq x) = P(Z \geq -x) = \Phi(x)$ too, since the standard normal is symmetric. Both cases give exactly $\Phi(x)$ for every $n$, so trivially $F_{X_n}(x) \to \Phi(x)$: yes, $X_n \xrightarrow{d} N(0,1)$. But individually, $X_n$ never settles at all — it swings deterministically between $Z$ and $-Z$ every single step, with $|X_{n+1}-X_n| = 2|Z|$ forever, never shrinking. The distribution's shape converges (trivially, since it's the same shape at every step) while the actual sequence of values does nothing resembling settling down — the cleanest possible illustration that convergence in distribution says nothing about individual paths.

What this means in practice

Every confidence interval built from "the sample mean is approximately normal for large $n$" is invoking convergence in distribution, not a claim that any individual sample mean has homed in on the truth — that stronger claim is what the law of large numbers provides instead, and it's a different, complementary statement. Backtest significance tests, asymptotic standard errors, and bootstrap confidence bands all lean on convergence in distribution results to justify treating a finite-sample statistic as approximately normal.