Sub-Gaussian and Sub-Exponential Tail Bounds

Estimating anything from data, a mean return, a regression coefficient, a covariance entry, comes with the question: how far could my estimate plausibly be from the truth, given a finite sample? The honest answer depends entirely on how "wild" the underlying data's tails are. Sub-Gaussian and sub-exponential are two named categories of tail behavior that let you write down a precise, provable bound on the chance of a large deviation, without needing to know the data's exact distribution, only that it belongs to one of these two well-behaved families.

An analogy: two kinds of well-mannered crowds

Picture a crowd where almost everyone stands close to average height, and the chance of finding someone freakishly tall falls off extremely fast the taller you look, that's the sub-Gaussian crowd, behaving like heights really do in nature, clustering tightly with a thin tail. Now picture a crowd of household incomes: still well-behaved and mostly clustered, but with a noticeably fatter tail, very high incomes are rarer than average incomes, but not nearly as rare as freakishly tall people are, and the tail thins out more gradually. That's the sub-exponential crowd. Both crowds are "well-behaved" in the sense that extreme values are genuinely rare and provably bounded, but the sub-exponential crowd's rare events are less rare than the sub-Gaussian crowd's, and any estimate built from sub-exponential data needs correspondingly more caution (or more data) before you can trust it as tightly.

The definitions, one symbol at a time

A random variable $X$ (mean zero, for simplicity) is sub-Gaussian with parameter $\sigma$ if its tail probability satisfies

In plain English: the chance of a deviation of size $t$ shrinks like $e^{-t^2}$, exactly the shape of a normal distribution's tail, bounded probability of extreme values that falls off very fast as $t$ grows, faster and faster the further out you go. A random variable is sub-exponential with parameters $(\sigma, b)$ if instead

decaying like $e^{-t}$ rather than $e^{-t^2}$, still exponentially fast, but linearly in $t$ in the exponent rather than quadratically, so large deviations are meaningfully more likely than under a sub-Gaussian bound of similar scale. In plain English: sub-exponential tails are heavier than sub-Gaussian ones (though still much lighter than, say, a Cauchy or power-law tail, which would have no such exponential bound at all), every sub-Gaussian variable is automatically sub-exponential, but not vice versa.

Worked example 1: comparing deviation bounds at t=5

For a sub-Gaussian variable with $\sigma=1$, the bound on $\mathbb{P}(|X|>5)$ is $2\exp(-25/2) = 2\exp(-12.5) \approx 7.5\times10^{-6}$, vanishingly small. For a sub-exponential variable with comparable scale ($b=1$), the bound on the same event is $2\exp(-5/2) = 2\exp(-2.5) \approx 0.164$, over 20,000 times larger. This single comparison shows concretely why the distinction matters in practice: an estimator built on sub-exponential noise (such as returns with occasional larger jumps, or squared/product terms like sample variances, which are inherently sub-exponential even when the underlying variable is sub-Gaussian) needs a substantially larger sample before a given confidence level can be claimed, compared to one built on genuinely sub-Gaussian noise.

Worked example 2: sample size needed for a target confidence

Suppose you want the sample mean of $n$ i.i.d. observations to be within $1 of the true mean with 95% confidence. If the underlying data is sub-Gaussian with $\sigma=5$, standard concentration bounds show $n$ on the order of $2\sigma^2\log(2/0.05) \approx 2(25)(3.69) \approx 185$ observations suffice. If instead the data is only sub-exponential with the same nominal scale, the required $n$ to achieve the same guarantee is typically several times larger, because the exponential-in-$t$ (rather than $t^2$) decay means the union-bound style argument needs a bigger buffer, a rough back-of-envelope figure would be several hundred to over a thousand observations, depending on the exact sub-exponential parameters. Knowing which regime your data lives in materially changes how much history you need before trusting a stated confidence interval.

What this means in practice

These two categories are the workhorse assumptions behind almost every finite-sample guarantee in modern high-dimensional statistics, bounds on how much a sample covariance matrix can deviate from the truth, how many observations the lasso needs to recover a sparse signal, or how tightly a sample mean concentrates. Return series are often modeled as sub-exponential rather than sub-Gaussian precisely because real market data has heavier tails than a normal distribution would suggest, and any theoretical guarantee derived assuming sub-Gaussian behavior can be badly miscalibrated if applied uncritically to genuinely sub-exponential or heavier-tailed data.