Logarithmic scoring rule Created 2026-10-07 Updated 2026-10-14
The logarithmic scoring rule rewards the natural logarithm of the reported probability mass or probability density at the observed outcome. The title uses the reward convention. For a binary report , its negative is the logarithmic loss:For this is ; for it is . At an endpoint, use these outcome-specific forms: assigning zero probability to the outcome incurs infinite loss, while assigning it probability one gives zero. The natural logarithm therefore makes a confidently wrong binary forecast far more costly than the bounded Brier score does.
Under the stipulated Bernoulli distribution with , the expected logarithmic loss isIts unique minimum is at , where the expected loss is approximately . At it is approximately . More generally, the excess expected logarithmic loss over truthful reporting is the Kullback-Leibler divergence in the order true law first, reported law second:For , this Kullback-Leibler divergence is nonnegative and vanishes only at , explaining why the logarithmic scoring rule is a strictly proper scoring rule. The same Kullback-Leibler divergence identity applies to probability density functions relative to a common measure when the truthful expected logarithmic loss is finite and the comparison expected value is well defined.[ref]
Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others. Section 3.1, Example 3, and Section 4.1
For sequential forecasts, the existing prequential log score identity explains how logarithmic rewards add over observations. Past exam of the mathematics course of the University of Cambridge 2013 iii Paper 34 4 d Solution Created 2026-10-03 Updated 2026-10-14
Model gives the sequential forecast . The probability chain rule yieldsApplication of Bayes' theorem makes the factors successive posterior predictive distributions, and their product is the Bayesian model evidence. Under conditionally independent sampling it isTaking logarithms gives the prequential log score identityTherefore the Bayes factor isThis solves the second half of part (d), which is missing from the TeX. Proper priors and finite positive evidences are needed; unrelated improper-prior normalizing constants do not cancel.