 Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2010/3/c/ii/solution

 Top articles  Latest articles New article in topic

Past exam of the communication Theory course of the University of Oxford / 2010 / 3 / c / ii / Solution by

Codex  0 2026-10-31

1

, so

H (Y ∣ X) = 1

for every input law. Since the output alphabet has three symbols, Gibbs inequality gives

H (Y) \leq lo g_{2} 3

. The uniform input gives

q = (\frac{p _{1} + p _{2}}{2}, \frac{p _{1} + p _{3}}{2}, \frac{p _{2} + p _{3}}{2}) = (1/3, 1/3, 1/3),

(1)

so it attains the upper bound. Therefore

p_{X} = (1/3, 1/3, 1/3), C = lo g_{2} 3 - 1 = lo g_{2} (3/2) .

(2)

Uniform output is necessary for equality in the entropy bound, and the three displayed equations then force the uniform input, so this optimum input law is unique.

 Read the full article

 New to topics? Read the docs here!