Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2006/iii/paper-30/3/a/solution

Write for the largest number of codewords in a binary length- code with minimum Hamming distance at least . The volume of a Hamming ball does not depend on its centre and is
For the Hamming bound, put . The radius- Hamming balls around distinct codewords are disjoint: a point belonging to two such balls would, by the triangle inequality for Hamming distance, put their centres at distance at most . Counting points in the binary cube therefore gives
For the Gilbert–Varshamov bound, choose a code maximal under inclusion among those with minimum Hamming distance at least , for example by successively adding any allowable word. Its radius- Hamming balls cover the cube. Otherwise an uncovered word would be at distance at least from every codeword and could be added, contradicting maximality. Thus
The lower bound is an existence assertion about a suitable code, while the upper bound holds for every such code. Together they give
Let be the binary entropy. For ,
The assumed Hamming ball volume exponent applies also to any integer radii with . Indeed, for each small these radii eventually lie between and . monotonicity of sandwiches its normalized logarithm between limits and ; continuity then lets tend to zero. Consequently the asymptotic Gilbert–Varshamov bound and asymptotic Hamming bound have respective exponential scales
Equivalently, the optimal code rate satisfies
These are bounds on the rate, not a claim that the optimal rate has a known limit or equals either endpoint.

New to topics? Read the docs here!