Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2008/3/a/solution

Unique decipherability makes the map from to the concatenated binary string injective. Entropy is unchanged by an injective relabelling, and memorylessness gives independence, hence
Given , the string has at most possibilities. The entropy bound for finite alphabets therefore gives . Averaging,
The random length cannot simply be discarded: it can also carry information. Let be the maximum codeword length, finite since the alphabet is finite. Since is a function of and takes at most values,
Divide by and let . This proves . The conclusion also covers a deterministic source; the source's lowercase in its final expectation denotes the random source letter.

New to topics? Read the docs here!