Past exam of the communication Theory course of the University of Oxford / 2011 / 1 / b / i

Table of contents
- Solution i

Solution

For the uniform distribution

u_{r} = 1/ n

, direct expansion gives

d (p_{X}, u) = \sum_{r} p_{X} (r) lo g_{2} p_{X} (r) + lo g_{2} n = lo g_{2} n - H (X) .

Part (a) therefore proves

H (X) \leq lo g_{2} n,

with equality exactly for

p_{X} (r) = 1/ n

for every

r

. Thus the uniform distribution is the unique entropy maximizer.

 Ancestors (12)

 View article source

 Discussion (0) New discussion

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.

  See all articles in the same topic Create my own version