NGBoost and Probabilistic Boosting

A standard gradient boosting model predicts a single number — a forecast return, a default probability, a price. NGBoost predicts an entire distribution instead: for a normal-distribution target, it outputs both a mean and a standard deviation for every observation, so a prediction comes with its own built-in uncertainty estimate rather than a bare point forecast.

Worked example. Forecasting next-quarter revenue growth for a set of companies, an ordinary boosting model might predict "6% growth" for every firm in a certain cluster, with no sense of which ones are riskier bets. NGBoost instead predicts, for one company, a mean of 6% with a standard deviation of 2% (a fairly confident forecast) and for another, a mean of 6% with a standard deviation of 9% (the same central guess, but far less reliable) — letting a portfolio manager size positions or set confidence bands using the spread, not just the point estimate.

The "natural gradient" part matters because a distribution's parameters (mean and spread) aren't independent of each other the way ordinary gradients assume; naively boosting each parameter separately can produce unstable updates. NGBoost corrects for this using the geometry of the probability distribution itself, which is why it converges reliably where a naive multi-output approach might not.