Large Deviations and Cramér's Theorem

The Law of Large Numbers says a sample average converges to the true mean. That's reassuring, but a risk manager needs more: not just "the average settles down," but how rare is it, precisely, for the average to sit far from where it should be after $n$ trials? "Very rare" isn't a number you can size capital against. Cramér's theorem gives the exact exponential rate at which that rareness decays — turning a vague intuition into a formula.

The analogy: the odds of an unlikely streak

If a coin is fair, seeing 60% heads in 10 flips is mildly surprising. Seeing 60% heads in 10,000 flips would be almost unbelievable — not just "more surprising," but surprising by an amount that compounds. Large deviations theory quantifies exactly how that improbability compounds: the probability of the sample average sitting at some wrong level $x$ instead of the true mean shrinks like $e^{-nI(x)}$, where $I(x)$ is a fixed "cost per observation" of drifting to $x$. Double the sample size, and you don't double the improbability — you square it.

Writing it down

Let $X_1, \ldots, X_n$ be i.i.d. with mean $\mu$, and let $\bar{X}_n = \frac1n\sum X_i$. Cramér's theorem says that for $x \ne \mu$,

$$\lim_{n\to\infty} \frac{1}{n} \log P(\bar{X}_n \approx x) = -I(x),$$

where $I(x)$, the rate function, is defined as the Legendre transform of the log-MGF:

$$I(x) = \sup_{s} \big( sx - \log M_X(s) \big).$$

In words: the probability of the sample average landing near some off-target value $x$ behaves like $e^{-nI(x)}$ for large $n$, and $I(x)$ — a single number per $x$, always $\ge 0$ and equal to zero exactly at $x=\mu$ — is computed by the same optimize-over-$s$ trick used in Chernoff bounds. In fact Cramér's theorem is the exact asymptotic version of what Chernoff bounds only bound from above: $I(x)$ is the tightest possible exponential rate, not just an upper bound on it.

Near the mean, $I(x)$ behaves roughly like a parabola — this plot's shape is a stand-in for that local behavior; the further $x$ sits from $\mu$, the steeper $I$ climbs and the more astronomically rare that deviation becomes.

Worked example 1: fair coin, rate function

For $X_i \in \{0,1\}$ Bernoulli($p=0.5$), the rate function works out to the binary relative entropy:

$$I(x) = x\log\frac{x}{0.5} + (1-x)\log\frac{1-x}{0.5}.$$

At $x = 0.6$: $I(0.6) = 0.6\log(1.2) + 0.4\log(0.8) = 0.6(0.182) + 0.4(-0.223) = 0.109 - 0.089 = 0.020$. So $P(\bar{X}_n \approx 0.6) \approx e^{-0.020n}$. At $n=100$: $e^{-2.0}\approx 0.135$ — plausible. At $n=10{,}000$: $e^{-200} \approx 10^{-87}$ — utterly impossible in practice. The same 10-percentage-point gap from the mean goes from "happens sometimes" to "will never happen" purely because $n$ grew, exactly the compounding the streak analogy described.

Watch the running average tighten around the truth as $n$ grows — large deviations theory is the precise statement of how fast the probability of it sitting away from that band collapses, exponentially in $n$ rather than merely shrinking.

Worked example 2: sizing a "never happens" loss

A market maker's daily P&L is the sum of 200 independent trades, each with mean edge \mu = \5$and roughly$\pm$50$range. Historical data suggests$I(x)$near a loss level of$\bar{x}=-$5per trade (i.e., total daily P&L around-$1{,}000$instead of the expected$+$1{,}000$) is about$I \approx 0.05$. Then$P(\text{daily P&L} \approx -$1{,}000) \approx e^{-0.05 \times 200} = e^{-10} \approx 4.5\times10^{-5}$ — roughly a 1-in-22,000 event, i.e. expect to see it about once every 85 years of trading days. That number is exactly the kind of figure a capital or limits committee needs, and it came from one estimated rate function, not from guessing "should be rare."

What this means in practice

• Setting risk limits. The rate function converts "how many standard deviations away" (a normal-distribution habit) into an honest, distribution-specific exponential probability — useful precisely when P&L isn't normal.
• Queueing and execution. The same machinery bounds the probability that an order queue or inventory position drifts far from target, informing how much buffer capital or inventory a market maker needs to hold.
• Insurance and ruin. Large deviations theory underlies ruin probability estimates: how unlikely is it that cumulative losses exceed reserves before some horizon.