Solution
= Solution
For finite-valued <random variables>, their <mutual information> is
$$
\boxed{I(X;Y)=H(X)+H(Y)-H(X,Y)=H(Y)-H(Y\mid X)=H(X)-H(X\mid Y).}
$$
Equivalently,
$$
I(X;Y)=\sum_{x,y}p(x,y)\log_2\frac{p(x,y)}{p_X(x)p_Y(y)},
$$
with zero joint-probability terms omitted. It is the <relative entropy> of the joint law from the product of its marginals and is nonnegative, with <equality> exactly when the two variables are <independent>.