 Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2008/1/b/solution

The conditional output distributions are permutations of the same triple, so both conditional entropies equal

H_{0} = H (1/6, 1/3, 1/2) .

p

, the three output probabilities are

q_{1} = (2 - p) /6

q_{2} = (1 + 2 p) /6

, and

q_{3} = (3 - p) /6

. Therefore

I (p) = - j = 1 \sum 3 q_{j} lo g_{2} q_{j} - H_{0} .

Differentiating, the constant terms cancel because

\sum q_{j}^{'} = 0

I^{'} (p) = \frac{1}{6} lo g_{2} \frac{( 2 - p ) ( 3 - p )}{( 1 + 2 p ) ^{2}}, I^{''} (p) = - \frac{1}{l o g _{e} 2} \sum_{j} \frac{( q _{j}^{'} ) ^{2}}{q _{j}} < 0.

Thus there is a unique maximum. Its stationary equation reduces to

3 p^{2} + 9 p - 5 = 0

, whose root in

[0, 1]

p_{m a x} = \frac{141 - 9}{6} .

The other root is negative; strict concavity proves this is the global maximum.

 New to topics? Read the docs here!