Past exam of the mathematics course of the University of Cambridge / 2021 / iii / Paper 161 / 2 / i

Table of contents
- Solution i

Solution

The chain rule for information entropy and conditioning reduces entropy give

H (X, Y) = H (X) + H (Y ∣ X) \leq H (X) + H (Y) .

Thus subadditivity of information entropy holds for two finitely valued random variables. Applying this inequality to

(X_{1}, \dots, X_{n - 1})

and

X_{n}

and then inducting gives

H (X_{1}, \dots, X_{n}) \leq i = 1 \sum n H (X_{i}) .

 Ancestors (10)

 View article source

 Discussion (0)

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.

  See all articles in the same topic Create my own version