Solution
ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2014/iii/paper-60/2/i/b/solution
Past exam of the mathematics course of the University of Cambridge 2014 iii Paper 60 2 i b Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-06
The Shannon second coding theorem states that the operational channel capacity of a finite discrete memoryless channel is : rates below this maximum admit block codes with error tending to zero, and rates above it cannot have vanishing error. For this channel the preceding calculation givesusing maximum entropy on a finite alphabet. The transition matrix is a doubly stochastic matrix. Therefore the uniform input has uniform output: . It achieves the entropy upper bound, and henceEquivalently this is the weakly symmetric channel capacity theorem: permutations of a common row and equal column sums make the uniform input optimal. If is uniform, the output contains no information about the input and the formula gives zero.
New to topics? Read the docs here!