 Probability calibration

Probability calibration means that reported probabilities agree with the frequencies of the outcomes they describe. For binary classification, let $Y\in \{0,1\}$ be the true label and let $P\in [0,1]$ be the predicted positive-class probability. Population calibration is the conditional expectation conditionAmong cases receiving a value near $0.8$, the positive fraction should therefore be near $0.8$, with uncertainty in any finite sample.^[ref]

A simple example separates calibration from discrimination. Suppose the population has a positive fraction of $0.3$ and a predictor always reports $P=0.3$. Then $\mathbb{E}[Y\mid P]=\mathbb{E}[Y]=0.3$, so it is calibrated, although it cannot rank one case above another. Conversely, replacing a useful score $s$ by a strictly increasing transformation such as $s^3$ preserves ranking but generally changes the claimed probability. Thus ordering cases and attaching reliable numerical probabilities are different tasks.

A reliability diagram estimates the agreement by grouping predictions and comparing each group's mean prediction with its observed outcomes. Assess it using suitable held-out observations. A calibrator fitted to predictions on the same observations used to fit the original model can inherit an overly optimistic picture of performance; fitting and evaluation need appropriate separation.^[ref]

A 2017 study found substantial confidence miscalibration in the evaluated neural networks, even when prediction accuracy improved. It also tested temperature scaling as a compact post-processing adjustment. These were empirical results on specified models and datasets, rather than a theorem that all neural networks are miscalibrated.^[ref]