Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2006/ii/paper-4/4g/solution

Shannon second coding theorem identifies the channel capacity of a discrete memoryless channel as : rates below admit block codes with error probability tending to zero, whereas a rate above cannot achieve vanishing error. Entropies here are measured in bits. For a binary erasure channel, a non-erased output determines its input exactly, while an erasure, independent of the input, leaves the original uncertainty unchanged. If , then
The binary entropy function has maximum one at . Therefore
This includes the noiseless case and the wholly erased case .

New to topics? Read the docs here!