 Brier score

The binary Brier score applies squared-error loss to a reported success probability $p$ and the observed indicator $y\in \{0,1\}$. It is a loss, so smaller is better. Its values lie in $[0,1]$: predicting the outcome with certainty gives zero, while assigning it zero probability gives one. Unlike the logarithmic scoring rule loss, the binary Brier score remains bounded even for a confidently wrong forecast.

For a Bernoulli distribution with true success probability $q$, averaging the two squared losses givesThe report controls only $(p-q{)}^2$; the remaining term is the variance of the Bernoulli random variable. Consequently the Brier score is a strictly proper scoring rule. At $q=0.7$, the expected loss is $(p-0.7{)}^2+0.21$, with unique minimum $0.21$ at $p=0.7$. Reporting $p=0.2$ instead gives expected loss $0.46$, despite its better loss on the particular outcome $Y=0$ above.

For several possible categories, the Brier score sums the squared differences between reported probabilities and observed category indicators. With two categories that sum is $2(p-y{)}^2$, so its numerical values are twice the binary convention used here, with the same optimizing report. Brier's original ^[ref] averages these category-sum losses over forecasts. Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others (Section 3.1, Example 1, and Table 1) gives the category-sum and binary forms as negative rewards; negating them gives the losses described here.