Quantile (Pinball) Loss for Neural Forecasting

A model trained with mean squared error learns to output one number: its best estimate of the average outcome. But a risk manager doesn't just want the average next-day return — they want to know "what's the worst plausible outcome I should size for," a specific quantile like the 5th percentile, not the mean. Getting a network to output a chosen quantile instead of the mean requires a different loss function that penalizes over-prediction and under-prediction by different amounts, in a ratio set by which quantile you want.

The analogy: a delivery company hedging against being late

A delivery company promising a "90% on-time" estimate isn't promising the average delivery time — it's promising a time that 90% of deliveries beat. To find that number, being early is barely penalized (arriving early is fine), but being late is penalized heavily and often (it should only happen 10% of the time). Tune that penalty ratio and you get a different promised time: a "50% on-time" estimate penalizes early and late symmetrically, landing back on the median. Quantile loss (also called pinball loss) is exactly this asymmetric penalty, made precise with a number.

The formula

For a target quantile $\tau \in (0,1)$ (e.g. $\tau = 0.9$ for the 90th percentile), residual $r = y - \hat{y}$:

$$L_\tau(r) = \begin{cases} \tau \cdot r & r \ge 0 \\ (\tau - 1) \cdot r & r < 0 \end{cases}$$

In words: when the model under-predicts ($r = y - \hat{y} \ge 0$, the true value came in above the forecast), the penalty is $\tau$ times the gap. When the model over-predicts ($r < 0$), the penalty is $(1-\tau)$ times the gap (since $\tau - 1$ is negative, multiplying it by a negative $r$ gives a positive loss). For $\tau = 0.9$, under-predicting is penalized 9 times more heavily than over-predicting by the same amount — which pushes the model's optimal output up, away from the mean, until only 10% of outcomes exceed it. That is exactly the defining property of the 90th percentile.

Worked example 1: computing pinball loss by hand at $\tau = 0.9$

Suppose the model forecasts $\hat{y} = 100$ and the true outcome is $y = 108$ (an under-prediction, $r = 8 \ge 0$): loss $= 0.9 \times 8 = 7.2$. Now suppose instead the true outcome is $y = 95$ (an over-prediction, $r = -5 < 0$): loss $= (0.9 - 1) \times (-5) = 0.5$. The same roughly-sized miss costs 14 times more when the model under-shot than over-shot — exactly the asymmetry needed to push the trained forecast upward toward the 90th percentile.

Worked example 2: recovering the median at $\tau = 0.5$

At $\tau = 0.5$: under-prediction penalty is $0.5 \times r$, over-prediction penalty is $(0.5-1)\times r = -0.5r$, which for negative $r$ gives $0.5|r|$ — the two cases are now penalized identically per unit of error. This symmetric case is exactly the ordinary mean absolute error loss, and minimizing it is well known to recover the median of the target distribution, not the mean. This confirms the formula behaves correctly at the one point where the answer is already familiar: $\tau=0.5$ is the special case of quantile loss that everyone already knows by another name.

What this means in practice

Several parallel output heads at different $\tau$ values, as above, give a full predictive interval from a single architecture — exactly what's needed for VaR-style risk estimates, execution cost bands, or any forecast where the tail matters as much as the center. A subtlety worth watching: quantile heads trained independently can cross (the 0.9 head predicting below the 0.5 head) since nothing in the loss enforces monotonicity across separately-trained quantiles.