Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2008/4/b/solution

The redundancy is , measured in bits per source symbol. Write for the total mass passing through interior node . Counting each codeword's length as one for each interior node its route visits gives
The entropy can be split into the conditional binary decisions along that route. Algebraically each node contributes
Summing telescopes: each non-root interior mass appears once positively as a parent and once negatively as a child. The root term is and the remaining leaf terms sum to . Hence
Subtracting this identity from the expected-length identity proves
Thus redundancy is the sum of the probability-weighted imbalances of the binary branch decisions. Nodes with no mass contribute zero by continuity. The formula holds for every full prefix tree, even without optimality.

New to topics? Read the docs here!