Chain rule for information entropy

For discrete random variables,

H (X, Y) = H (X) + H (Y ∣ X) .

Iterating gives

H (X_{1}, \dots, X_{n}) = \sum_{i} H (X_{i} ∣ X_{1}, \dots, X_{i - 1})

.

 Ancestors (6)

 Incoming links (9)

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 164 / 1 / i / Solution
Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 164 / 2 / Solution
Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 164 / 3 / ii / Solution
Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 164 / 3 / i / Solution
Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 164 / 4 / i / Solution
Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 1 / c / Solution
Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 4 / d / Solution
Shearer's inequality
Subadditivity of information entropy

 View article source

 Discussion (0)

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.

  See all articles in the same topic Create my own version