Past exam of the mathematics course of the University of Cambridge / 2023 / ii / Paper 1 / 3I (Coding and Cryptography)

A binary code $c:\mathcal{A}\to \{0,1{\}}^{\ast }$ is a prefix code when no codeword is a proper prefix of another. If the source letters have probabilities $p_1,\dots ,p_m$ and codeword lengths $l_1,\dots ,l_m$, the expected value of its codeword length isIt is an optimal prefix code when it minimizes $L$ among all binary prefix codes for that source.

Suppose $p_i>p_j$ but $l_i>l_j$. Exchanging the codewords assigned to ${\mu }_i$ and ${\mu }_j$ leaves the set of words, and hence prefix-freeness, unchanged. The change in expected length iscontradicting optimality. Thuswhich is the probability monotonicity of optimal prefix-code lengths.

Represent the code by its binary prefix tree and choose a codeword $wb$ of maximal length, where $b\in \{0,1\}$. Its sibling is $w(1-b)$. If that sibling were not a codeword, it could not have a codeword below it either, since such a word would be longer than $wb$. Replacing $wb$ by $w$ would then preserve prefix-freeness and strictly reduce $L$. This is impossible for an optimal code, so $w(1-b)$ is also a maximal-length codeword. The pair differs only in its last digit, proving the deepest sibling property of an optimal prefix code.

The Huffman coding merges may be chosen asResolving the tied weights so that the original $0.4$ symbol is merged only at the final step gives the codeIts lengths are $(1,3,3,3,3)$ and its expected length isThe Huffman coding theorem proves optimality, so an optimal coding with all but one word of the same length does exist. This is an instance of a nonunique optimal prefix code: a different resolution of the ties also gives the optimal profile $(2,2,2,3,3)$.

An optimal code cannot have five distinct lengths. By the deepest sibling property of an optimal prefix code, at least two maximal-length codewords have the same length. Thus the answers are respectively