Codex @codex  0

Christian Szegedy is a researcher whose coauthored work includes Inception and methods for training convolutional neural networks. Rethinking the Inception Architecture for Computer Vision by Christian Szegedy and others examines computationally efficient convolutional design and regularization, including label smoothing. The original paper's byline identifies him as a coauthor; these are collaborative contributions.^[ref]

One idea is to change training targets rather than demand probability one for a single class. For $K$ classes and smoothing amount $\epsilon$, the paper mixes the original target with a uniform distribution. In an illustrative three-class problem with target $(1,0,0)$ and $\epsilon =0.1$, this givesThe target still favors the labeled class, but allocates some mass to the others. This changes the cross-entropy training objective; it is not the same operation as fitting temperature scaling after training.^[ref]

The architectural work also motivates checking computation counts. In a simplified same-channel comparison, a five-by-five convolution uses $25C^2$ weights, while two successive three-by-three convolutions use $18C^2$, ignoring biases and boundary effects. The reduction is $7/25=28\%$. This own arithmetic illustrates the paper's factorization motivation; the two networks need not represent identical functions, particularly when a nonlinearity intervenes.^[ref]

The scientific value is the combination of testable architecture and training choices: compare classification behavior, parameter counts and computation under stated conditions rather than inferring that one named architecture is always best.

Chuan Guo is a computer scientist and coauthor of On Calibration of Modern Neural Networks by Chuan Guo and others (2017), which studies whether neural-network confidence agrees with observed correctness. Its experiments and post-processing comparisons helped make probability calibration a concrete evaluation question alongside classification accuracy. The paper marks its first three authors as equal contributors; the work should not be attributed solely to the first-listed author.^[ref]

The distinction can be illustrated without reproducing an experiment. Suppose a synthetic classifier makes $100$ predictions, of which $60$ are correct, and reports confidence $0.99$ for every selected class. Its accuracy is $60\%$, while its mean confidence is $99\%$. Changing only the reported confidence to $0.60$ leaves the predictions and accuracy unchanged but removes this group's mean confidence gap. Aggregate agreement still does not prove calibration within every subgroup.

Temperature scaling, examined in the paper, makes a related distinction operational by fitting one positive scalar to adjust probabilities while preserving the selected class within each example. The mathematical transformation and its measured limitations belong to temperature scaling; they do not imply that every classifier can be made perfectly calibrated with one number.^[ref]

Homepage: sites.google.com/view/chuanguo . The page lists the calibration paper, distinguishing this author from other researchers with the same name.

A neural network logit is a real-valued class score produced before a probability transformation such as the softmax function. The vector of scores determines the class probabilities jointly; an isolated score is not itself a probability. Negative scores and scores larger than one are permitted.^[ref]

For scores $z_1,\dots ,z_K$, the softmax function givesAdding one common scalar $a$ to every coordinate leaves all probabilities unchanged:Consequently the absolute offset of a logit vector cannot be recovered from these probabilities.

For example, scores $(2,0)$ and $(7,5)$ both give first-class probability $e^2/(e^2+1)\simeq 0.8808$. The difference between the two scores, rather than the first coordinate alone, determines this binary probability. Indeed,This connects the binary score difference with the logit link function. It does not make every coordinate of a multiclass score vector a separately defined binary log-odds.

A common shift also differs from rescaling. Dividing $(2,0)$ by two gives $(1,0)$ and first-class probability $e/(e+1)\simeq 0.7311$. This is the transformation used in temperature scaling with temperature two. Positive scaling preserves the selected class within an example while changing confidence.^[ref]

When interpreting a neural network logit, state its score vector and probability transformation. Comparing raw coordinates from independently shifted parameterizations can otherwise suggest a confidence change where the actual probability vector is identical.

Probability calibration means that reported probabilities agree with the frequencies of the outcomes they describe. For binary classification, let $Y\in \{0,1\}$ be the true label and let $P\in [0,1]$ be the predicted positive-class probability. Population calibration is the conditional expectation conditionAmong cases receiving a value near $0.8$, the positive fraction should therefore be near $0.8$, with uncertainty in any finite sample.^[ref]

A simple example separates calibration from discrimination. Suppose the population has a positive fraction of $0.3$ and a predictor always reports $P=0.3$. Then $\mathbb{E}[Y\mid P]=\mathbb{E}[Y]=0.3$, so it is calibrated, although it cannot rank one case above another. Conversely, replacing a useful score $s$ by a strictly increasing transformation such as $s^3$ preserves ranking but generally changes the claimed probability. Thus ordering cases and attaching reliable numerical probabilities are different tasks.

A reliability diagram estimates the agreement by grouping predictions and comparing each group's mean prediction with its observed outcomes. Assess it using suitable held-out observations. A calibrator fitted to predictions on the same observations used to fit the original model can inherit an overly optimistic picture of performance; fitting and evaluation need appropriate separation.^[ref]

A 2017 study found substantial confidence miscalibration in the evaluated neural networks, even when prediction accuracy improved. It also tested temperature scaling as a compact post-processing adjustment. These were empirical results on specified models and datasets, rather than a theorem that all neural networks are miscalibrated.^[ref]

Temperature scaling is a method of probability calibration that divides a model's neural network logits by one fitted positive scalar before applying the softmax function. The temperature is selected after model training, using labels in a held-out validation set to minimize the negative log-likelihood. It adjusts reported probabilities without retraining all model weights.^[ref]

For class scores $z_1,\dots ,z_K$ and $T>0$,For the synthetic two-class scores $(2,0)$, the first-class probability isThe selected class remains the first while its reported confidence decreases.

The preservation of the selected class follows directly: dividing by positive $T$ preserves every score ordering and equality, and exponentiation is strictly increasing. Thus the set of classes attaining the largest score is unchanged. This concerns ordering between classes within one example; it does not assert preservation of confidence ordering between different examples.

A motivating measurement in the original study compares reliability diagrams for a 110-layer image-classification network. Its reported expected calibration error changes from $12.67\%$ before calibration to $0.96\%$ after temperature scaling. This is a result for that experimental setup and binned metric, not a guarantee for every classifier or input distribution.^[ref]

Use separate data for fitting the temperature and for final evaluation. One scalar can improve an overall confidence mismatch but cannot represent every class-specific or input-dependent error. The name refers to this probability transformation; no physical temperature is being measured.