Past exam of the communication Theory course of the University of Oxford / 2011 / 1 / b / ii

Table of contents
- Solution ii

Solution

Normalize the candidate distribution by setting

q_{r} = c 2^{- r}, c = \frac{1}{1 - 2 ^{- n}} = \frac{2 ^{n}}{2 ^{n} - 1} .

The supplied finite sum gives its mean

\sum_{r = 1}^{n} r q_{r} = c (2 - (n + 2) 2^{- n}) = \frac{2 ^{n + 1} - ( n + 2 )}{2 ^{n} - 1} = m .

Thus it meets the constraint. Since

lo g_{2} q_{r} = lo g_{2} c - r

, every other distribution

p

with mean

m

satisfies

d (p, q) = - H (p) - lo g_{2} c + m = H (q) - H (p) .

Nonnegativity and the equality case in part (a) prove

p_{Y} (r) = \frac{2 ^{n}}{2 ^{n} - 1} 2^{- r}, H (Y)_{m a x} = m - lo g_{2} \frac{2 ^{n}}{2 ^{n} - 1} .

The maximum is unique; the entropy comparison uses the fixed mean rather than an unconstrained optimization.

 Ancestors (12)

 View article source

 Discussion (0) New discussion

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.

  See all articles in the same topic Create my own version