Chain rule for information entropy
= Chain rule for information entropy
For discrete random variables,
$$
H(X,Y)=H(X)+H(Y\mid X).
$$
Iterating gives $H(X_1,\ldots,X_n)=\sum_iH(X_i\mid X_1,\ldots,X_{i-1})$.
= Chain rule for information entropy
For discrete random variables,
$$
H(X,Y)=H(X)+H(Y\mid X).
$$
Iterating gives $H(X_1,\ldots,X_n)=\sum_iH(X_i\mid X_1,\ldots,X_{i-1})$.