Asymptotic Normality of Estimators

You fit a model and get a single number for a parameter — say, a regression slope or a maximum-likelihood volatility estimate. You want an error bar on it, but you don't know the exact distribution of your estimator; deriving it exactly can be genuinely hard or impossible. Asymptotic normality is the escape hatch used everywhere in statistics: for almost any well-behaved estimator, no matter its exact finite-sample distribution, its error becomes approximately bell-shaped once your sample is large enough — and "approximately bell-shaped" is all you need to build a confidence interval.

An analogy: many small votes add up to one predictable shape

Think of your estimation error as the sum of many small, individually messy nudges — each data point contributing a little push toward or away from the truth. Any one nudge might have a weird, lopsided distribution. But add up hundreds of them, and the specific weirdness of each individual nudge washes out; what survives is only their combined scale. This is the same mechanism as the Central Limit Theorem for a plain average, except here it's applied to the error of an estimator — a maximum-likelihood estimate, a regression coefficient, a sample correlation — rather than to a raw average of data.

The statement, one symbol at a time

Let $\hat\theta_n$ ("theta-hat, from $n$ observations") be your estimator and $\theta$ the true parameter value. Asymptotic normality says

$$\sqrt{n}\,(\hat\theta_n - \theta) \xrightarrow{d} N(0, \, V)$$

Read this left to right: take the estimation error $\hat\theta_n - \theta$, blow it up by $\sqrt{n}$ (to compensate for the fact that the raw error is shrinking toward zero), and as $n$ grows, this rescaled error's distribution — not its value — settles into a normal curve centered at zero with some variance $V$ (called the asymptotic variance). In plain English: your estimator's error is, for large samples, approximately normally distributed around the truth, with a spread that shrinks like $1/\sqrt{n}$.

Practically, this is used in its unpacked, working form:

$$\hat\theta_n \;\overset{\cdot}{\sim}\; N\!\left(\theta, \; \frac{V}{n}\right)$$

The dot over the tilde means "approximately distributed as" — this is the sentence "$\hat\theta_n$ behaves like a normal random variable centered on the truth, with variance $V/n$," which is exactly what lets you write $\hat\theta_n \pm 1.96\sqrt{V/n}$ as a 95% confidence interval, whatever $\hat\theta_n$ actually is.

Set the parent distribution above to something skewed and drag the sample size up. The individual data points stay skewed forever — but the distribution of the sample mean (itself a simple estimator) becomes visibly bell-shaped well before the parent distribution does. That is asymptotic normality happening in front of you, for the simplest possible estimator.

The running-average explorer above shows the companion fact: as the bell curve above tightens, the running estimate itself is homing in on the truth at the $1/\sqrt n$ rate baked into the asymptotic-normality formula — the shrinking band you see there is literally $\sqrt{V/n}$ narrowing as $n$ grows.

Worked example 1: the asymptotic variance of a maximum-likelihood estimate

Suppose you estimate the parameter $\lambda$ of an exponential distribution — modelling, say, the time between large trades — by maximum likelihood, from $n=100$ observations, getting $\hat\lambda = 0.5$. For the exponential distribution, likelihood theory gives the asymptotic variance as $V = \lambda^2$ (this comes from the model's Fisher information, a separate topic, but the number is standard and derivable).

Plug in: $V = 0.5^2 = 0.25$. The standard error of $\hat\lambda$ is then $\sqrt{V/n} = \sqrt{0.25/100} = \sqrt{0.0025} = 0.05$.

A 95% confidence interval: $0.5 \pm 1.96 \times 0.05 = 0.5 \pm 0.098$, so roughly $(0.40, 0.60)$. Notice what happened: you never derived the exact finite-sample distribution of $\hat\lambda$ (which is a scaled gamma distribution, more awkward to work with) — you used the large-sample normal approximation instead, and it produced a usable, standard interval in three lines.

Worked example 2: doubling the sample shrinks the interval by $\sqrt{2}$, not by half

Keep the same setup, but now with $n=400$ observations instead of 100 (four times as many), and suppose $\hat\lambda$ is still $0.5$.

Standard error: $\sqrt{V/n} = \sqrt{0.25/400} = \sqrt{0.000625} = 0.025$ — exactly half of the previous $0.05$, because quadrupling $n$ shrinks the standard error by $\sqrt{4}=2$.

95% interval: $0.5 \pm 1.96 \times 0.025 = 0.5 \pm 0.049$, or roughly $(0.45, 0.55)$ — half the width of the $n=100$ interval. This is the concrete meaning of "$\sqrt{n}$" in the formula: precision buys you diminishing returns, since it takes a 4x larger sample to halve your error bar, not a 2x larger one.

What this means in practice

• This is why confidence intervals and $t$-statistics are attached to almost any fitted parameter, from a regression coefficient to a GARCH volatility parameter to a factor loading — none of these have simple, exactly-known finite-sample distributions, but virtually all are asymptotically normal.
• "Large enough" sample size is doing real work and is rarely checked. Asymptotic normality is a limit statement; with $n=20$ noisy observations the true error distribution may still be visibly skewed or heavy-tailed, and the normal-based interval can be badly wrong even though the formula runs without complaint.
• The delta method extends this result to any smooth transformation of an asymptotically normal estimator — which is how a volatility, a Sharpe ratio, or an odds ratio built from an asymptotically normal input inherits its own asymptotically normal error bar.