The binary entropy function is the information entropy of a Bernoulli random variable:with endpoint values zero. It is symmetric under , reaches one bit at , and hasin the interior. Its strict concavity expresses why an even mixture of two outcomes has more uncertainty than a strong bias toward either one. The curve in mutual information also shows how this same uncertainty becomes the loss term in binary symmetric channel capacity.
New to topics? Read the docs here!