Binary entropy (source code)

= Binary entropy
{title2=$h_2(p)$}

The binary entropy function is the <information entropy> of a <Bernoulli random variable>:
$$
h_2(p)=-p\log_2p-(1-p)\log_2(1-p),\qquad 0\leq p\leq1,
$$
with endpoint values zero. It is symmetric under $p\mapsto1-p$, reaches one <bit> at $p=1/2$, and has
$$
h_2'(p)=\log_2\frac{1-p}{p},\qquad h_2''(p)=-\frac1{(\ln2)p(1-p)}<0
$$
in the interior. Its strict <concavity> expresses why an even mixture of two outcomes has more uncertainty than a strong bias toward either one. The curve in <mutual information> also shows how this same uncertainty becomes the loss term in <binary symmetric channel capacity>.