Past exam of the chemistry course of the year 4 of the natural sciences course of the University of Cambridge / 2022 / Paper 3 / Section F / 47 / c

For $K$ mutually exclusive classes, the output layer should provide $K$ nonnegative probabilities summing to one, with labels aligned consistently to output coordinates. A softmax activation on logits $a_k$ providesSubtracting ${\max }_{\ell }{a}_{\ell }$ from all logits before exponentiation preserves these probabilities and prevents numerical overflow. Categorical cross-entropy $-{\sum }_k{y}_k\log p_k$ is appropriate for one-hot or probabilistic class targets. Independent sigmoid outputs fit a different setting in which multiple labels can be present simultaneously and their probabilities need not sum to one.