Probability Calibration

A gradient-boosted tree can rank trades from most to least likely to be profitable extremely well, and still be badly calibrated — its "0.9" outputs might really win only 60% of the time. Ranking accuracy and calibration are different properties. A model only needs to get the order right to rank well; it needs the actual number to mean something to be calibrated, and most classifiers, tree ensembles especially, are optimized for the former and get the latter almost by accident.

Two standard fixes

Platt scaling fits a logistic regression with a single input — the base model's raw score — to the true 0/1 outcomes: $p = \frac{1}{1 + e^{-(aS + b)}}$, where $S$ is the base model's score and $a, b$ are the two numbers this tiny model learns. In words: it stretches or squashes the score with a sigmoid curve until predicted frequencies line up with real ones. It works well when the miscalibration is a smooth, S-shaped distortion, which tree ensembles typically produce because their outputs are averages of many 0/1 leaf votes and get pulled toward the middle.

Isotonic regression instead fits a non-decreasing step function directly to the data — no assumed shape, just "output must never decrease as the raw score increases." It's more flexible and can fix distortions Platt scaling can't, but needs more data to avoid overfitting the calibration mapping itself, since it has many more effective parameters.

Worked example

A random forest predicting whether a trade idea will be profitable outputs a raw score of 0.80 for a particular setup. Backtesting shows that among all past setups where the raw model said "around 0.80," the actual win rate was only 0.62 — the tree votes are compressed toward the middle less than true frequencies would suggest at the high end, and the raw score overstates confidence. Fitting Platt scaling on a held-out calibration set of 5,000 past predictions yields $a = 1.4$, $b = -0.55$. Applying it to the raw score of 0.80 (as a logit-scale input, $S = \ln\frac{0.80}{0.20} = 1.386$): $p = \frac{1}{1+e^{-(1.4 \times 1.386 - 0.55)}} = \frac{1}{1+e^{-1.39}} \approx 0.80$. In this example the correction happens to land back near the raw score at that particular point even though the curve has shifted elsewhere — the point is that the mapping is fit from data, not assumed to be the identity.

What this means in practice

Calibration matters whenever the number is used downstream, not just the ranking — sizing a position in proportion to win probability, setting a Kelly fraction, or feeding a probability into an expected-value calculation all break if 0.7 doesn't really mean 70%. It matters less for a model only used to rank candidates and pick the top decile, where only the ordering is used.