 Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2011/1/b/ii/solution

Normalize the candidate distribution by setting

q_{r} = c 2^{- r}, c = \frac{1}{1 - 2 ^{- n}} = \frac{2 ^{n}}{2 ^{n} - 1} .

The supplied finite sum gives its mean

\sum_{r = 1}^{n} r q_{r} = c (2 - (n + 2) 2^{- n}) = \frac{2 ^{n + 1} - ( n + 2 )}{2 ^{n} - 1} = m .

Thus it meets the constraint. Since

lo g_{2} q_{r} = lo g_{2} c - r

, every other distribution

p

with mean

m

satisfies

d (p, q) = - H (p) - lo g_{2} c + m = H (q) - H (p) .

Nonnegativity and the equality case in part (a) prove

p_{Y} (r) = \frac{2 ^{n}}{2 ^{n} - 1} 2^{- r}, H (Y)_{m a x} = m - lo g_{2} \frac{2 ^{n}}{2 ^{n} - 1} .

The maximum is unique; the entropy comparison uses the fixed mean rather than an unconstrained optimization.

 New to topics? Read the docs here!