Temperature and Beta Calibration

Many classifiers, especially large neural networks and deep boosted ensembles, output probabilities that are systematically overconfident: a model that says "95% confident" might actually be right only 80% of the time. Temperature scaling is the simplest fix — it divides the model's raw pre-probability scores (logits) by a single learned constant, the temperature $T$, before converting them to probabilities, softening overconfident predictions without changing which class is predicted.

Worked example. A neural network's raw output for one example gives class probabilities of [0.97, 0.02, 0.01] — extremely confident. Fitting a temperature $T = 2.5$ on a validation set (chosen because it minimizes validation log-loss) and dividing the logits by $T$ before the softmax pulls those probabilities toward [0.81, 0.11, 0.08] — same predicted class, but a confidence level that better matches how often the model is actually correct on similar examples. Beta calibration extends this with a second parameter, letting the correction curve bend asymmetrically, which helps when overconfidence is worse at one end of the probability range (e.g., near-certain predictions) than the other.

Both methods are attractive because they need almost no extra data or computation — a small validation set and a one- or two-parameter fit — and because rescaling doesn't touch the model's ranking of predictions, so accuracy and ranking metrics like AUC are completely unaffected; only the reported probabilities change.