Quadratic Discriminant Analysis

Linear discriminant analysis (LDA) separates classes with a straight boundary because it assumes every class's data is spread out the same way, just centered differently. That's often wrong — "calm market" days might cluster tightly while "stressed" days spread out much more widely, in a different shape. Quadratic discriminant analysis (QDA) drops the shared-spread assumption and lets each class have its own spread, at the cost of a curved boundary.

The analogy: one umbrella size versus custom umbrellas

LDA assumes everyone in a group needs the same size and shape of umbrella, just centered over a different spot. QDA measures each person and gives them a custom-shaped umbrella that fits their build — more accurate, but you now need far more information per person, and if you only have a few examples of some group, their custom umbrella is a noisy guess.

The mechanics: separate covariance per class

Both model each class $k$ as multivariate normal with mean $\mu_k$, classifying by whichever class makes a point most probable. LDA forces every class to share one covariance matrix $\Sigma$, which cancels out of the comparison between classes, leaving a boundary linear in $x$. QDA instead estimates a separate $\Sigma_k$ per class, so that cancellation doesn't happen:

$$\delta_k(x) - \delta_j(x) = -\tfrac{1}{2}\log|\Sigma_k| - \tfrac{1}{2}(x-\mu_k)^\top \Sigma_k^{-1}(x-\mu_k) + \tfrac{1}{2}\log|\Sigma_j| + \tfrac{1}{2}(x-\mu_j)^\top \Sigma_j^{-1}(x-\mu_j).$$

In plain English: the boundary is now genuinely quadratic in $x$ — it can curve or bend around a class, because each class's own shape directly warps the boundary near it instead of canceling out.

Worked example: two regimes with different volatility

"Calm" days: mean volatility 12%, SD 3%. "Stressed" days: mean 25%, SD 10% — higher on average and far more variable. Under LDA's shared-variance assumption you'd be forced to average these two spreads, distorting both. Under QDA, a new observation at 18% is 2 SDs from calm (using SD 3) but only 0.7 SDs from stressed (using SD 10) — QDA correctly weighs stressed's wider tolerance, something a single shared spread cannot represent.

Toggle between a linear and more flexible boundary above: allowing curvature lets the classifier hug an irregularly-shaped class more closely — the visual version of what QDA's per-class covariance buys over LDA's shared one.

What this means in practice

QDA needs enough data per class to estimate a full covariance matrix — roughly $p^2/2$ parameters per class. With few observations in some class, its estimated $\Sigma_k$ is noisy, and the curved boundary can overfit despite looking more flexible on paper. QDA helps most when classes visibly differ in spread, not just location, with enough data in every class.