Mixture Distributions

Daily stock returns don't look like one normal distribution — they look like a normal distribution most days, punctuated by occasional much wilder days. Fitting a single normal distribution to the whole history forces one compromise variance that's too wide for calm days and too narrow for crisis days. A mixture distribution doesn't compromise: it says the data is secretly generated by more than one underlying distribution, with some probability of drawing from each, and lets you fit both honestly rather than blending them into one bad average.

The analogy: two vending machines behind one slot

Imagine a single coin slot that's secretly wired to two different vending machines behind the wall — most of the time (say 90%) it dispenses from the "calm" machine, and occasionally (10%) from the "chaotic" machine which gives much more variable output. From the front, you just see one stream of results, and if you compute their overall average and spread, you get numbers that don't match either machine individually — the combined output is a mixture, and understanding it requires modeling the two machines separately, not fitting one distribution to the blended stream.

Writing it down

A mixture distribution with $K$ components has density

$$f(x) = \sum_{k=1}^{K} \pi_k \, f_k(x),$$

where each $f_k$ is a component density (often normal) and the weights $\pi_k \ge 0$ sum to 1. In words: pick a component $k$ with probability $\pi_k$, then draw $x$ from that component's distribution — the overall density is the weighted average of the component densities, not a single distribution with blended parameters. The mixture's overall mean and variance follow the law of total expectation and variance:

$$E[X] = \sum_k \pi_k \mu_k, \qquad \text{Var}(X) = \sum_k \pi_k \left(\sigma_k^2 + \mu_k^2\right) - \left(\sum_k \pi_k \mu_k\right)^2.$$

In words: the mixture's variance is not just the weighted average of the component variances — it also picks up extra spread from the components having different means, which is exactly the mechanism by which mixing two normal distributions with different means or variances produces fat tails, or even two visible humps, that no single normal distribution could ever have.

Picture blending this curve (a calm regime, small spread) with a second, much wider normal curve centered near the same place but rarely sampled — the resulting mixture has a sharp central peak (from the calm component dominating most days) with unusually heavy shoulders (from the rare wide component), a shape no single normal curve, however you stretch it, can reproduce.

Worked example 1: two-regime return model

Model daily returns as a 90/10 mixture: calm regime $N(0.05\%, 0.8\%^2)$ with weight 0.9, stressed regime $N(-0.5\%, 3.5\%^2)$ with weight 0.1. Overall mean: $0.9(0.05\%) + 0.1(-0.5\%) = 0.045\% - 0.05\% = -0.005\%$. Overall variance: $0.9(0.8\%^2 + 0.05\%^2) + 0.1(3.5\%^2 + (-0.5\%)^2) - (-0.005\%)^2 \approx 0.9(0.6425) + 0.1(12.5) - 0.0000 = 0.578 + 1.25 = 1.828$ (in %² units), so overall standard deviation $\approx 1.35\%$. A single normal fit to this data would report a 1.35% daily vol — but that number describes neither regime: it overstates the 0.8% calm-day risk on 90% of days and drastically understates the 3.5% stressed-day risk on the 10% of days that actually matter most for tail risk.

Worked example 2: how a mixture creates fat tails from two thin-tailed pieces

Using the same mixture, what's $P(\text{return} < -5\%)$? Under the (wrong) single-normal fit with mean $\approx 0\%$, sd $1.35\%$: $z = -5/1.35 = -3.70$, giving $P \approx 0.0001$ (about 1 in 10,000 days). Under the correct mixture: this event is essentially only reachable via the stressed regime, $N(-0.5\%, 3.5\%^2)$, where $z = (-5-(-0.5))/3.5 = -1.29$, giving $P\approx 0.099$ within that regime — so overall $P(\text{return}<-5\%) \approx 0.1 \times 0.099 + 0.9\times(\text{negligible}) \approx 0.0099$, about 1 in 100 days. The mixture says this crash-sized move is roughly 100 times more likely than the single-normal fit claims — this gap, entirely from ignoring regime structure, is a large part of why normal-based VaR models have historically underestimated crash risk.

Sample several batches of paths here — some batches will look calm and tightly clustered, others will throw an outlier path that strays far from the rest. That batch-to-batch inconsistency is a live picture of mixing: no single volatility number describes every batch equally well, which is exactly the regime-mixture story above.

What this means in practice

• Regime-aware risk models. Fitting a Gaussian mixture (via the EM algorithm) to return data, rather than a single normal, is a standard and tractable way to capture fat tails without abandoning the normal distribution's convenient math entirely.
• Stress testing. Mixture models make the stressed component explicit and separately calibratable — you can ask "what if the stressed regime's probability doubles" as a clean, interpretable scenario.
• Related to regime-switching. A mixture model is the static (unconditional) cousin of a hidden Markov model — the latter additionally models how regimes persist and transition over time, rather than treating each day's regime draw as independent. See Hidden Markov Models.