= Solution
The <chain rule for information entropy> converts the two differences into <conditional entropies>. Their gap, in the order right minus left, is
$$
\begin{aligned}
&\bigl[H(X,Z)-H(X)\bigr]-\bigl[H(X,Y,Z)-H(X,Y)\bigr]\\
&\qquad=H(Z\mid X)-H(Z\mid X,Y)=I(Y;Z\mid X)\geq0.
\end{aligned}
$$
This is <additional conditioning reduces entropy>. The <relative entropy> representation of <conditional mutual information> identifies exactly when that gap vanishes: \b[equality holds if and only if $Y$ and $Z$ have <conditional independence> given $X$.] Knowledge of $Y$ then changes no <conditional probability> of $Z$ after $X$ is known.
Back to article page
