Loss function

A loss function assigns a numerical penalty to a prediction and its observed outcome. Its expected value is the population risk optimized by a statistical learning procedure.

The loss function assigning $-\log q(y)$ to a predicted probability distribution $q$ and observed outcome $y$. It is the negative logarithmic scoring rule; its expected value under the true probability distribution is cross-entropy. Predicting zero probability for an observed outcome gives infinite loss.

The population risk of a prediction rule is its expected loss on a fresh observation from the target probability distribution.