 Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2011/1/b/iii/solution

The given logarithmic moment is

ℓ = \frac{2}{n ( n + 1 )} \sum_{r = 1}^{n} r lo g_{2} r,

where the

r = 1

term is zero. It is attained by the positive distribution

q_{r} = \frac{2 r}{n ( n + 1 )},

whose probabilities sum to one. Write

A = 2/ [n (n + 1)]

, so

lo g_{2} q_{r} = lo g_{2} A + lo g_{2} r

. For any distribution

p

with the required logarithmic moment,

d (p, q) = - H (p) - lo g_{2} A - ℓ = H (q) - H (p) .

Consequently the unique maximizer and its entropy are

p_{Z} (r) = \frac{2 r}{n ( n + 1 )}, H (Z)_{m a x} = lo g_{2} \frac{n ( n + 1 )}{2} - ℓ .

The increasing weights are consistent with the prescribed moment; they need not be a decreasing power law.

 New to topics? Read the docs here!