Past exam of the communication Theory course of the University of Oxford / 2011 / 1 / b

For the uniform distribution $u_r=1/n$, direct expansion givesPart (a) therefore proveswith equality exactly for $p_X(r)=1/n$ for every $r$. Thus the uniform distribution is the unique entropy maximizer.

Normalize the candidate distribution by settingThe supplied finite sum gives its meanThus it meets the constraint. Since ${\log }_2{q}_r={\log }_{2}c-r$, every other distribution $p$ with mean $m$ satisfiesNonnegativity and the equality case in part (a) proveThe maximum is unique; the entropy comparison uses the fixed mean rather than an unconstrained optimization.

The given logarithmic moment iswhere the $r=1$ term is zero. It is attained by the positive distributionwhose probabilities sum to one. Write $A=2/[n(n+1)]$, so ${\log }_2{q}_r={\log }_{2}A+{\log }_{2}r$. For any distribution $p$ with the required logarithmic moment,Consequently the unique maximizer and its entropy areThe increasing weights are consistent with the prescribed moment; they need not be a decreasing power law.