Past exam of the mathematics course of the University of Cambridge 2015 iii Paper 66 4 iii Solution Created 2026-10-03 Updated 2026-10-06
Use the Shannon second coding theorem in its channel capacity form . The finite-alphabet erasure channel erases independently of the input. On an unerased output Bob knows exactly; on the erasure output his posterior distribution is still . Its classical conditional entropy is thereforeEquivalently, and , so the binary entropy terms cancel in the mutual information.
For an alphabet of size , Shannon entropy is at most , with equality for the uniform distribution. Maximizing the displayed mutual information proves the capacity of a finite-alphabet erasure channel:This includes , which transmits the whole symbol, and , which transmits no information.