Sub-Gaussian and Sub-Exponential Tail Bounds

Estimating anything from data — a mean return, a regression coefficient, a covariance entry — comes with the question: how far could my estimate plausibly be from the truth, given a finite sample? The honest answer depends entirely on how "wild" the underlying data's tails are. Sub-Gaussian and sub-exponential are two named categories of tail behavior that let you write down a precise, provable bound on the chance of a large deviation, without needing to know the data's exact distribution — only that it belongs to one of these two well-behaved families.

An analogy: two kinds of well-mannered crowds

Picture a crowd where almost everyone stands close to average height, and the chance of finding someone freakishly tall falls off extremely fast the taller you look — that's the sub-Gaussian crowd, behaving like heights really do in nature, clustering tightly with a thin tail. Now picture a crowd of household incomes: still well-behaved and mostly clustered, but with a noticeably fatter tail — very high incomes are rarer than average incomes, but not nearly as rare as freakishly tall people are, and the tail thins out more gradually. That's the sub-exponential crowd. Both crowds are "well-behaved" in the sense that extreme values are genuinely rare and provably bounded — but the sub-exponential crowd's rare events are less rare than the sub-Gaussian crowd's, and any estimate built from sub-exponential data needs correspondingly more caution (or more data) before you can trust it as tightly.

The definitions, one symbol at a time

A random variable $X$ (mean zero, for simplicity) is sub-Gaussian with parameter $\sigma$ if its tail probability satisfies

$$\mathbb{P}(|X| > t) \le 2\exp\left(-\frac{t^2}{2\sigma^2}\right) \quad \text{for all } t > 0.$$

In plain English: the chance of a deviation of size $t$ shrinks like $e^{-t^2}$, exactly the shape of a normal distribution's tail — bounded probability of extreme values that falls off very fast as $t$ grows, faster and faster the further out you go. A random variable is sub-exponential with parameters $(\sigma, b)$ if instead

$$\mathbb{P}(|X| > t) \le 2\exp\left(-\frac{t}{2b}\right) \quad \text{for } t \text{ large enough,}$$

decaying like $e^{-t}$ rather than $e^{-t^2}$ — still exponentially fast, but linearly in $t$ in the exponent rather than quadratically, so large deviations are meaningfully more likely than under a sub-Gaussian bound of similar scale. In plain English: sub-exponential tails are heavier than sub-Gaussian ones (though still much lighter than, say, a Cauchy or power-law tail, which would have no such exponential bound at all) — every sub-Gaussian variable is automatically sub-exponential, but not vice versa.

Worked example 1: comparing deviation bounds at t=5

For a sub-Gaussian variable with $\sigma=1$, the bound on $\mathbb{P}(|X|>5)$ is $2\exp(-25/2) = 2\exp(-12.5) \approx 7.5\times10^{-6}$ — vanishingly small. For a sub-exponential variable with comparable scale ($b=1$), the bound on the same event is $2\exp(-5/2) = 2\exp(-2.5) \approx 0.164$ — over 20,000 times larger. This single comparison shows concretely why the distinction matters in practice: an estimator built on sub-exponential noise (such as returns with occasional larger jumps, or squared/product terms like sample variances, which are inherently sub-exponential even when the underlying variable is sub-Gaussian) needs a substantially larger sample before a given confidence level can be claimed, compared to one built on genuinely sub-Gaussian noise.

Worked example 2: sample size needed for a target confidence

Suppose you want the sample mean of $n$ i.i.d. observations to be within $1 of the true mean with 95% confidence. If the underlying data is sub-Gaussian with $\sigma=5$, standard concentration bounds show $n$ on the order of $2\sigma^2\log(2/0.05) \approx 2(25)(3.69) \approx 185$ observations suffice. If instead the data is only sub-exponential with the same nominal scale, the required $n$ to achieve the same guarantee is typically several times larger, because the exponential-in-$t$ (rather than $t^2$) decay means the union-bound style argument needs a bigger buffer — a rough back-of-envelope figure would be several hundred to over a thousand observations, depending on the exact sub-exponential parameters. Knowing which regime your data lives in materially changes how much history you need before trusting a stated confidence interval.

What this means in practice

These two categories are the workhorse assumptions behind almost every finite-sample guarantee in modern high-dimensional statistics — bounds on how much a sample covariance matrix can deviate from the truth, how many observations the lasso needs to recover a sparse signal, or how tightly a sample mean concentrates. Return series are often modeled as sub-exponential rather than sub-Gaussian precisely because real market data has heavier tails than a normal distribution would suggest, and any theoretical guarantee derived assuming sub-Gaussian behavior can be badly miscalibrated if applied uncritically to genuinely sub-exponential or heavier-tailed data.