= Equality in data processing inequality
{title2=$I(X;Z)=I(X;Y)\iff I(X;Y\mid Z)=0$}
Under the finite <Markov chain> hypothesis $X\to Y\to Z$, the gap identity in the <data processing inequality> gives
$$
I(X;Z)=I(X;Y)\quad\Longleftrightarrow\quad I(X;Y\mid Z)=0.
$$
By the zero criterion for <conditional mutual information>, equality means that $X$ and $Y$ have <conditional independence> given $Z$. After seeing the processed output, the original observation provides no additional information about $X$. Similarly, equality in the other bound, $I(X;Z)=I(Y;Z)$, holds exactly when $I(Y;Z\mid X)=0$.
Equality does not require retaining every detail of $Y$. Let $X,W$ be <independent> fair <bits>, put $Y=(X,W)$, and let the processed output be $Z=X$, the first coordinate of $Y$. This is a <Markov chain>, and
$$
I(X;Y)=H(X)-H(X\mid X,W)=1-0=1,
\qquad I(X;Z)=I(X;X)=1.
$$
Also $I(X;Y\mid Z)=H(X\mid X)-H(X\mid X,W)=0$. The output discards the entire unrelated <bit> $W$, reducing $H(Y)=2$ to $H(Z)=1$, while preserving all <mutual information> about $X$. This is what makes equality useful for choosing a compact observation that retains all information about the quantity of interest.
Back to article page
