Past exam of the mathematics course of the University of Cambridge / 2006 / ii / Paper 4 / 4G / Solution

Shannon second coding theorem identifies the channel capacity of a discrete memoryless channel as $C={\max }_{P_X}I(X;Y)$: rates below $C$ admit block codes with error probability tending to zero, whereas a rate above $C$ cannot achieve vanishing error. Entropies here are measured in bits. For a binary erasure channel, a non-erased output determines its input exactly, while an erasure, independent of the input, leaves the original uncertainty unchanged. If $\mathbb{P}(X=1)=q$, thenThe binary entropy function has maximum one at $q=1/2$. ThereforeThis includes the noiseless case $p=0$ and the wholly erased case $p=1$.