Linear probability score Created 2026-10-07 Updated 2026-10-14
The linear probability score pays the reported probability of whichever binary outcome occurs. With report and observed , its reward isIt looks sensible: assign more probability to what happens and receive more reward. But the expected value under a Bernoulli distribution with true success probability isThis is an affine function of the report. When , the maximum over is at ; when , it is at . At , every report ties. Thus the linear probability score encourages reporting certainty about the more likely outcome and is not a proper scoring rule.
For the stipulated , the expected value of the reward is . Honest reporting earns in expectation, whereas earns . The forecaster improves expected reward by replacing a 70% forecast with a 100% forecast. The Brier score and logarithmic scoring rule avoid this incentive, as the expected-score diagram under proper scoring rule shows.
Past exam of the mathematics course of the University of Cambridge 2013 iii Paper 34 4 b Solution Created 2026-10-03 Updated 2026-10-07
Let be the genuine rain probability and the reported probability. The linear probability score has expected rewardThis is linear in , so its optimal report and expected reward areAt every report scores on average. Truthful reporting scores , strictly below the optimum for interior . Thus the rule is not proper: it rewards maximal confidence in the more likely outcome. The expected total over days is .
Proper scoring rule Created 2026-10-07 Updated 2026-10-14
A proper scoring rule makes reporting the true probability distribution optimal in expected value, for every true probability distribution in the allowed class of forecasts. In the reward convention, the inequality in the title holds for every report and true law . The expected value is always taken under , because the report does not change how the outcome is generated. In the loss convention the same definition readsThis requirement makes a proper scoring rule useful for eliciting probabilities and comparing predictive distributions: deliberately distorting a known forecast cannot improve its expected reward or expected loss. The reward convention and its strict version are set out in Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others (Section 1).
For the Bernoulli distribution with , the Brier score has expected loss , and the negative logarithmic scoring rule has expected loss . Both are minimized at the honest report . The linear probability score, by contrast, maximizes its expected reward at . These are statements about averaging over both possible outcomes with their actual probabilities.
Expected losses of the Brier score and logarithmic scoring rule favor ; the linear probability score rewards
. Details. The left panel displays each excess expected loss divided by that rule's expected loss at : for the Brier score and approximately for the logarithmic scoring rule. This normalization permits comparison of curve shapes and preserves the minimizing report, since it subtracts a constant and divides by a positive constant independent of . Zero means no excess expected loss. The right panel shows the unmodified linear probability score reward, with its maximizing direction labeled explicitly.
Scoring rule Created 2026-10-07 Updated 2026-10-14
A scoring rule turns a probability distribution forecast and an observed outcome into a number. For example, a forecast of rain with probability should express uncertainty more usefully than a bare prediction of rain. A scoring rule lets us compare such forecasts after observations arrive, and its choice determines whether exaggerating confidence pays. A proper scoring rule rewards an honest account of uncertainty in expected value.
There are two sign conventions. A reward is maximized; a loss function is minimized. The conversion reverses every comparison. Here the linear probability score is a reward, while the binary Brier score and the negative logarithmic scoring rule are losses. Calling all three a "score" without specifying the direction would hide the central difference.
Throughout the binary examples, the random variable has a Bernoulli distribution with actual success probability , and the report assigns probability to and to . The value is stipulated by this statistical model; it is not inferred from data or a claim that a forecasting system is calibrated. In practice the true predictive distribution is unknown, and the relevant is the conditional probability of success given the information available when the forecast is issued. Propriety evaluates incentives for reporting that predictive distribution; it does not supply the unknown probabilities.
