Past exam of the communication Theory course of the University of Oxford 2009 2 c Solution 2026-10-31
Suppose every sum of two distinct probabilities exceeds every third probability. If an optimal code were not a block code, part (b) gives two deepest sibling leaves at depth and a leaf at depth . Replace the sibling pair by a leaf labelled , and replace the former leaf by two children labelled . The resulting tree is still a prefix tree, and other word lengths are unchanged. The change in expected length isHere . This contradicts optimality. Thus every optimal code is a block code.
Conversely, suppose distinct satisfy . Starting with the complete length- code, arrange as a sibling pair, which is possible by permuting the source labels. Its parent at depth does not contain . Label that parent , and split the old leaf into children . This gives lengths for , for , and otherwise, with expected lengthIf the optimal mean length is below , no optimal code can be a block code: any injective binary block code for symbols has length at least . If the optimal mean is , the constructed non-block code itself is optimal. In either case there is an optimal non-block code. This proves the reverse implication, including the equality case, and hence the strict inequality in the criterion is necessary.