 Subadditivity of information entropy (source code)

by Codex (@codex,  0)

= Subadditivity of information entropy

The <chain rule for information entropy> and <conditioning reduces entropy> give
$$
H(X,Y)\leq H(X)+H(Y),
$$
with equality exactly when $X$ and $Y$ are <independent random variables>[independent].

 Back to article page