Past exam of the mathematics course of the University of Cambridge 2007 iii Paper 31 1 d Solution Created 2026-10-03 Updated 2026-10-14
Suppose a leaf has depth . By part a there is a sibling pair of leaves at depth . These leaves are disjoint from the shallow leaf and its descendants, since a leaf has no descendants. Modify the prefix tree of a code by splitting the shallow leaf into two leaves at depth and replacing the deepest sibling pair by their parent as one leaf at depth . The number of leaves remains , so assign the equiprobable source letters bijectively to the new leaves.
The sum of depths for the three affected leaves changes from to . The change isAll other depths are unchanged. Since probabilities are equal, this strictly decreases the expected codeword length, contradicting the optimality of the Huffman code. This is the balancing exchange for a uniform optimal prefix code. There is therefore no leaf of depth at most , provingThe argument applies to every optimal binary prefix tree for the uniform source, independently of how Huffman ties are resolved.
Redundancy of a binary prefix code 2026-10-31
The redundancy of a binary prefix code is its expected codeword length minus the Shannon entropy of its source. It measures the excess number of binary digits per source symbol above the information lower bound. For a full prefix tree of a code, let be the probability of visiting an interior node and let be the conditional probability of taking its zero branch. Thenwhere is binary entropy. The first identity counts one digit for each visited interior node. The second follows by expanding : the interior terms telescope, leaving the negative sum of leaf probability logarithms. The final identity follows by subtraction. It makes nonnegativity immediate and proves that zero redundancy occurs exactly when each positive-mass branch decision is balanced. Consequently the leaf probabilities then equal .