 Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2007/3/b/solution

C \geq 0

. The two conditional-Shannon entropy expressions give

I (X; Y) = H (X) - H (X ∣ Y) = H (Y) - H (Y ∣ X) \leq min (H (X), H (Y)) .

Each Shannon entropy is bounded by the base-two logarithm of its alphabet size. Maximizing over the input distribution therefore gives

0 \leq C \leq lo g_{2} m

for the smaller size

m

 New to topics? Read the docs here!