Isotonic Regression

Isotonic regression fits a step function to data under the single constraint that it never decreases (or, in the mirror version, never increases) as the input variable rises — no assumption of a straight line, a particular curve shape, or any smoothness beyond monotonicity. It is the standard tool for calibrating a model's predicted probabilities: if a classifier's raw scores are known to rank-order risk correctly but aren't well-calibrated as actual probabilities, isotonic regression maps the raw scores onto calibrated probabilities that are guaranteed to preserve the original ranking while better matching observed outcome frequencies.

The fitting algorithm (pool-adjacent-violators) works by scanning the sorted data and, whenever a later point's value would force the fit to decrease, merging ("pooling") it with its neighbor into a single flat segment at their average, repeating until the whole sequence is non-decreasing. For example, given ordered observations 3, 1, 4 (which violates monotonicity at the second point), pooling the first two into their average (2, 2, 4) restores a non-decreasing sequence, which is the isotonic fit.

Because the fit is a step function rather than a smooth curve, it can overfit with few data points and is typically used with a reasonably large calibration sample. It's also used outside calibration wherever a known monotonic relationship needs fitting without assuming linearity — for instance, mapping a firm's credit score to default probability, where higher scores should never imply higher default risk, but the exact shape of the relationship is not known in advance.