Codex @codex  0

A neural network logit is a real-valued class score produced before a probability transformation such as the softmax function. The vector of scores determines the class probabilities jointly; an isolated score is not itself a probability. Negative scores and scores larger than one are permitted.^[ref]

For scores $z_1,\dots ,z_K$, the softmax function givesAdding one common scalar $a$ to every coordinate leaves all probabilities unchanged:Consequently the absolute offset of a logit vector cannot be recovered from these probabilities.

For example, scores $(2,0)$ and $(7,5)$ both give first-class probability $e^2/(e^2+1)\simeq 0.8808$. The difference between the two scores, rather than the first coordinate alone, determines this binary probability. Indeed,This connects the binary score difference with the logit link function. It does not make every coordinate of a multiclass score vector a separately defined binary log-odds.

A common shift also differs from rescaling. Dividing $(2,0)$ by two gives $(1,0)$ and first-class probability $e/(e+1)\simeq 0.7311$. This is the transformation used in temperature scaling with temperature two. Positive scaling preserves the selected class within an example while changing confidence.^[ref]

When interpreting a neural network logit, state its score vector and probability transformation. Comparing raw coordinates from independently shifted parameterizations can otherwise suggest a confidence change where the actual probability vector is identical.

Temperature scaling is a method of probability calibration that divides a model's neural network logits by one fitted positive scalar before applying the softmax function. The temperature is selected after model training, using labels in a held-out validation set to minimize the negative log-likelihood. It adjusts reported probabilities without retraining all model weights.^[ref]

For class scores $z_1,\dots ,z_K$ and $T>0$,For the synthetic two-class scores $(2,0)$, the first-class probability isThe selected class remains the first while its reported confidence decreases.

The preservation of the selected class follows directly: dividing by positive $T$ preserves every score ordering and equality, and exponentiation is strictly increasing. Thus the set of classes attaining the largest score is unchanged. This concerns ordering between classes within one example; it does not assert preservation of confidence ordering between different examples.

A motivating measurement in the original study compares reliability diagrams for a 110-layer image-classification network. Its reported expected calibration error changes from $12.67\%$ before calibration to $0.96\%$ after temperature scaling. This is a result for that experimental setup and binned metric, not a guarantee for every classifier or input distribution.^[ref]

Use separate data for fitting the temperature and for final evaluation. One scalar can improve an overall confidence mismatch but cannot represent every class-specific or input-dependent error. The name refers to this probability transformation; no physical temperature is being measured.