Quantile Regression Forests

A random forest that predicts "tomorrow's return is +0.3%" is throwing away useful information: it says nothing about how confident to be, or what the bad-case outcome looks like. A trader usually cares more about the 5th percentile — the plausible bad day — than the average. Quantile regression forests get that whole distribution almost for free, by changing what the forest remembers about each leaf, not how it is grown.

The analogy: don't just average the class, keep the whole roster

Picture a teacher predicting a student's exam score by finding the 50 most similar past students (same study habits, same background) and averaging their scores. That gives one number. But if the teacher instead kept the full list of those 50 past scores, she could also say "10% of similar students scored below 60" or "90% scored below 88" — the entire spread, not just its center. An ordinary regression forest is the average of the neighbors; a quantile regression forest is built from the same neighbors but reports the full ranked list.

How it works on top of an ordinary forest

A standard random forest grows many trees, and for a new point $x$, each tree routes it down to one leaf, and the leaf's average of training-set outcomes becomes that tree's prediction; averaging across trees gives the forest's final number. Quantile regression forests grow the exact same trees the exact same way, but at prediction time they do something different: instead of averaging the outcomes that landed in each relevant leaf, they collect all of those outcomes into one pooled list (weighting each training point by how often it lands in the same leaf as $x$, across all trees), and then read off any percentile of that pooled list.

$$\hat{F}(y \mid x) = \sum_{i=1}^{n} w_i(x)\, \mathbb{1}[y_i \le y]$$

In plain English: build a weighted tally of "how much does training example $i$ resemble the new point $x$" (from how often they co-occur in the same leaves), then use those weights to build a full cumulative distribution of the outcome, not just its mean. The $q$-th quantile is simply the value of $y$ where that cumulative distribution crosses $q$.

Worked example 1: a 5-tree toy forest

Suppose a new point $x$ lands in leaves containing these training outcomes across 5 trees: Tree 1: {2, 3, 5}, Tree 2: {3, 4, 5}, Tree 3: {2, 4}, Tree 4: {3, 5, 6}, Tree 5: {4, 5}. An ordinary forest averages each tree's leaf mean, then averages those: roughly $3.33, 4.0, 3.0, 4.67, 4.5$, averaging to about $3.9$. A quantile forest instead pools all 13 values — {2,2,3,3,3,4,4,4,5,5,5,5,6} — sorts them, and reads off, say, the 10th percentile (around 2) and the 90th percentile (around 5.4 by interpolation). The single "average" prediction of 3.9 hides that the plausible range is roughly 2 to 5.4.

Worked example 2: an asymmetric bad case

For a strategy return dataset, suppose the pooled outcomes for a given feature setting are mostly clustered around +1% but with a long left tail: {-4, -1, 0.5, 1, 1, 1.2, 1.5, 2}. The mean is about 0.28% — unremarkable. The 10th percentile, however, is close to -4%, revealing a real left-tail risk the mean-only prediction completely hides. This is exactly the case where reporting a single average would understate downside risk relative to reporting the 10th percentile alongside it.

Drag the parameters above and watch percentiles move independently of the mean — that is the extra information a quantile forest recovers from the same trees an ordinary forest would only average.

What this means in practice

Quant uses include sizing positions off a downside quantile rather than an expected value, and building prediction intervals around a forecast without assuming a symmetric, Gaussian-shaped error. It costs almost nothing extra over training an ordinary forest, since the same trees are reused.