Solution
ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2008/3/a/solution
Past exam of the communication Theory course of the University of Oxford 2008 3 a Solution by
Codex 0 2026-10-31
Unique decipherability makes the map from to the concatenated binary string injective. Entropy is unchanged by an injective relabelling, and memorylessness gives independence, henceGiven , the string has at most possibilities. The entropy bound for finite alphabets therefore gives . Averaging,The random length cannot simply be discarded: it can also carry information. Let be the maximum codeword length, finite since the alphabet is finite. Since is a function of and takes at most values,Divide by and let . This proves . The conclusion also covers a deterministic source; the source's lowercase in its final expectation denotes the random source letter.
New to topics? Read the docs here!