 Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2011/1/b/i/solution

u_{r} = 1/ n

, direct expansion gives

d (p_{X}, u) = \sum_{r} p_{X} (r) lo g_{2} p_{X} (r) + lo g_{2} n = lo g_{2} n - H (X) .

Part (a) therefore proves

H (X) \leq lo g_{2} n,

with equality exactly for

p_{X} (r) = 1/ n

for every

r

. Thus the uniform distribution is the unique entropy maximizer.

 New to topics? Read the docs here!