Past exam of the information theory course of the University of Oxford / 2021 / 3 / b

Write $T=g(Y)$. Because $T$ is determined by $Y$, the chain rule for mutual information givesThis is the data processing inequality. It holds for every input probability distribution. Maximizing each side over that same set of input probability distributions gives $C_{g(Y)}\le C_Y$; processing cannot increase channel capacity.

A simple sufficient condition for equality is that $g$ be injective on the reachable output alphabet. Then $Y$ can be recovered from $T$, giving $H(Y\mid T)=0$ and hence $I(X;Y\mid T)=0$ for every input law. The input that achieves the original channel capacity therefore achieves it after processing as well. Values of $g$ outside reachable outputs are irrelevant.

Injectivity is sufficient rather than necessary. More precisely, for finite alphabets equality in capacity data processing holds exactly when there is an original capacity-achieving input law for which $X$ and $Y$ have conditional independence given $T$. Sufficiency follows from the displayed gap identity. For necessity, maximize $I(X;T)$ at an input law $p_{\ast }$; if its maximum equals $C_Y$, then $C_Y=I_{p_{\ast }}(X;T)\le I_{p_{\ast }}(X;Y)\le C_Y$, so both inequalities are equalities. Processing may discard output details unrelated to the input without losing channel capacity.