Pinball Loss and Quantile Networks

Predicting the average outcome is the wrong goal whenever the actual question is about risk — a risk manager rarely cares "what's the expected loss," they care "what loss will only be exceeded 5% of the time." A pinball-loss quantile network is trained to answer that second kind of question directly, producing a threshold value for a chosen percentile rather than a single average guess.

The analogy: setting a price you're okay being wrong about, one-sided

Imagine setting a price for buying insurance against a bad outcome, where you're comfortable being wrong on one side (paying slightly more than needed) but not on the other (being caught badly underinsured). If you want to be right roughly 90% of the time that your threshold isn't exceeded, you should deliberately set it a bit high — being wrong low is far more costly than being wrong high. Quantile regression trains a model to find exactly that threshold: the value $q_\tau(x)$ such that a target fraction $\tau$ of outcomes fall below it, for whichever $\tau$ (say 0.1, 0.5, or 0.9) you choose.

The pinball loss

$$L_\tau(y, q) = \begin{cases} \tau (y - q) & \text{if } y \ge q \\ (1-\tau)(q - y) & \text{if } y < q \end{cases}$$

In words: if the actual outcome $y$ comes in above the predicted quantile $q$ (an undershoot), the penalty is $\tau$ times the gap; if it comes in below $q$ (an overshoot), the penalty is $(1-\tau)$ times the gap. For $\tau = 0.9$, undershooting is penalized nine times more heavily than overshooting per unit of error, which is exactly why the network learns to set $q$ high enough that only 10% of outcomes exceed it — that's the only way to make the heavily-weighted undershoot case rare. Setting $\tau=0.5$ recovers ordinary median regression, where both directions cost the same.

Worked example 1: the asymmetry in numbers

Let $\tau = 0.9$, true outcome $y = 10$. Compare a prediction $q=8$ (undershoot, $y > q$) against $q=12$ (overshoot, $y < q$).

$$L_{0.9}(10, 8) = 0.9 \times (10-8) = 1.8, \qquad L_{0.9}(10, 12) = 0.1 \times (12-10) = 0.2$$

Both predictions miss by the same amount, 2 units, but undershooting costs nine times as much as overshooting under $\tau=0.9$. That asymmetry is precisely what pushes a trained model toward the high side — a network minimizing average pinball loss across many such examples learns to systematically bias its predictions upward until only about 10% of true outcomes exceed them.

Worked example 2: fitting a median versus a 90th percentile

Five residuals from a small dataset: $\{-2, -1, 0, 3, 6\}$. For $\tau=0.5$ (median), the loss-minimizing constant prediction is the sample median, $0$: total pinball loss $= 0.5(2+1+0+3+6) = 6.0$. For $\tau=0.9$, the loss-minimizing constant is the empirical 90th percentile, roughly $5$ (between $3$ and $6$, closer to $6$): checking $q=5$, undershoots are $\{-2,-1,0,3\}$ contributing $0.9\times(5-(-2)+5-(-1)+5-0+5-3)=0.9\times(7+6+5+2)=18.0$, and the one overshoot $6$ contributes $0.1\times(6-5)=0.1$, total $18.1$ — moving $q$ toward $6$ trades a small increase in the rare overshoot penalty against a larger decrease across all four undershoots, which is exactly the calculation the training loss does automatically at every gradient step.

The two linear arms of the pinball loss meet at $q=y$ with different, asymmetric slopes — steeper on the side the loss punishes harder — which is the whole mechanism forcing the model away from the ordinary mean.

What this means in practice

Quantile networks are the direct neural-network route to Value-at-Risk-style forecasts and prediction intervals used for position sizing and stop placement — training separate output heads at $\tau = 0.05, 0.5, 0.95$ gives a lower bound, median, and upper bound from one model, without assuming the residuals are Gaussian, which the older mean-plus-fixed-standard-deviation approach implicitly does.