= Binary cross-entropy
{title2=$H(r,q)=-r\log_2q-(1-r)\log_2(1-q)$}
Let $Y$ follow a <Bernoulli distribution> with true event <probability> $r$, and let the model report event <probability> $q$. The <binary cross-entropy> is its expected <logarithmic loss>:
$$
H(r,q)=-r\log_2q-(1-r)\log_2(1-q).
$$
For $0<r<1$, it is minimized at $q=r$, where the value is <binary entropy> $h_2(r)$. An individual outcome has loss $-\log_2q$ when $Y=1$ and $-\log_2(1-q)$ when $Y=0$; the <binary cross-entropy> averages those two cases using the true <probabilities>.
For $r=1/4$, truthful reporting gives $h_2(1/4)\approx0.811278$ <bits>. Reporting $q=1/2$ gives one <bit>. Reporting $q=0.01$ gives
$$
H(1/4,0.01)=\tfrac14\log_2 100-\tfrac34\log_2 0.99\approx1.671839\text{ bits}.
$$
Both $q=1/4$ and $q=0.01$ select zero if one predicts only the more likely binary outcome. Their predicted <probabilities> are very different, however, and <cross-entropy> reveals the cost of underestimating the event that occurs a quarter of the time. Endpoint losses are computed using the realized outcome rather than multiplying zero by an undefined <logarithm>.
\Image[/binary-cross-entropy-forecast-loss.png]
{title=The expected logarithmic loss is minimized by reporting the true event probability}
{description=The true <Bernoulli distribution> has parameter $r=1/4$. The blue curve averages the orange event loss and gray nonevent loss with weights $1/4$ and $3/4$. Its minimum is at $q=r$, while both endpoint limits diverge because both outcomes have positive <probability>. These are calculated expectations, rather than measurements.}
Back to article page
