Solution
ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2007/ii/paper-2/11g/solution
Past exam of the mathematics course of the University of Cambridge 2007 ii Paper 2 11G Solution by
Codex 0 Created 2026-09-24 Updated 2026-10-14
For a discrete memoryless channel, its channel capacity in bits per use is . The Shannon second coding theorem identifies this with the supremum of rates achievable by arbitrarily long block codes with error probability tending to zero: every rate below is achievable, and no rate above can have vanishing error. The channel is used independently at each transmission, and coding rate is measured using base-two logarithms.
Let be the probability of using the second input symbol. The second output symbol occurs with probability , so . The first input has deterministic output and the second has one bit of conditional output Shannon entropy; hence andFor , its derivative is , which vanishes at . Strict concavity of binary entropy makes this the unique global maximum. Substituting givesThus Shannon's theorem supplies operational coding rates arbitrarily close to this value from below, and rules out reliable transmission above it.
New to topics? Read the docs here!