Quantile Regression

Ordinary regression is obsessed with the average. OLS fits a line through the mean of the outcome, and that's often the wrong question. A risk manager doesn't care about the average trading day, they care about the bad tail, the 5th-percentile day. A logistics team sizing capacity cares about the busy 95th-percentile hour, not the typical one. Quantile regression fits a line (or surface) through any percentile you choose, so you can model the tails directly instead of guessing at them from a mean and a standard-deviation assumption.

The payoff is that it shows how the whole shape of the outcome responds to your predictors, not just its center. If the spread of returns widens in high-volatility regimes, OLS misses it entirely, its single line just tracks the middle. Quantile regression draws one line per percentile, and when those lines fan apart, you're seeing the spread change.

The idea: swap the loss function

OLS minimises squared errors, which is what makes it target the mean (the mean is the number that minimises squared distance to a set of points). To target a percentile instead, you swap in an asymmetric penalty, the "check" or "pinball" loss. To fit the $\tau$-th quantile (say $\tau = 0.9$ for the 90th percentile), you minimise

$$\sum_i \rho_\tau\big(y_i - \hat y_i\big), \qquad \rho_\tau(u) = \begin{cases} \tau\, u & u \ge 0 \\ (\tau - 1)\, u & u < 0. \end{cases}$$

In words: an under-prediction (the actual came in above your line, $u > 0$) is charged at weight $\tau$, while an over-prediction is charged at weight $1 - \tau$. For $\tau = 0.9$ that's a $9\!:\!1$ penalty ratio, undershooting the truth is nine times as costly as overshooting. To dodge that expensive side, the fitted line is pushed up until only about $10\%$ of points sit above it, which is precisely the 90th percentile. Setting $\tau = 0.5$ makes the penalty symmetric and you're fitting the median.

Worked example: median vs mean with an outlier

Take five outcomes with no predictor, so we're just fitting a single constant: $\{1, 2, 3, 4, 100\}$. Fitting the mean (OLS with only an intercept) gives $\bar y = 110/5 = 22$, a number that describes none of the five values, dragged wildly by the outlier. Fitting the median (quantile regression at $\tau = 0.5$) gives $3$, the middle value, unmoved by how extreme the outlier is. Push the $100$ to $1{,}000$ and the OLS answer leaps to $202$ while the median stays put at $3$.

That robustness is the first reason to reach for quantile regression, but the deeper one is what happens with a predictor: fit $\tau = 0.05$ against volatility and you get a line for the conditional 5th percentile of returns, a direct, assumption-light estimate of Value at Risk (VaR) that lets the tail respond to conditions instead of scaling a fixed number by a normal quantile.

Where it misleads and what to watch

• Quantile crossing. Fit several percentiles separately and their lines can occasionally cross, implying, absurdly, that the 90th percentile sits below the 50th at some $x$. It's an artifact of fitting each independently; joint methods or monotonic constraints fix it.
• Noisier in the tails. Estimating the 1st percentile means leaning on the handful of most extreme points, so far-tail quantile fits have wide error bars. You need a lot of data to nail a deep tail.
• No neat closed form. Unlike OLS there's no $(X^\top X)^{-1}X^\top y$; fitting is a linear program. That's cheap now, but standard errors usually come from a bootstrap rather than a formula.
• Interpretation shifts. A coefficient is the effect of a predictor on that quantile, and it can differ across quantiles, that's the feature, not a bug, but it means "the effect of x on y" is no longer a single number.

The one-sentence summary: OLS answers "how does the average move?", quantile regression answers "how does the whole distribution move?". For anything tail-sensitive, risk limits, capacity, worst-case planning, that second question is the one that actually matters, which is why quantile regression sits close to Value at Risk (VaR) and Expected Shortfall (CVaR) in a risk toolkit.