Past exam of the information theory course of the University of Oxford / 2021 / 3 / c / i / Solution

Translate the output alphabet by $-\min (z_1,z_2)$ and put $d=|z_2-z_1|\ge 1$. The reachable outputs becomeFor $d=1$ the two sets overlap in two points; for $d=2$ in one; for $d\ge 3$ they are disjoint. In particular $|\mathcal{Y}|\ge 4$, and choosing noise values $0,1$ attains four.

With rows indexed by inputs $0,1,2$ and columns by outputs $0,1,2,3$, independence and uniform two-point noise giveEvery conditional output distribution has information entropy one bit, so $I(X;Y)=H(Y)-1\le {\log }_{2}4-1=1$. The input probability mass function $(1/2,0,1/2)$ makes all four outputs equiprobable, attaining that bound. ThusUsing a zero-probability input in a channel-capacity maximization is legitimate; it is unrelated to the strictly positive source weights needed for the tree-shape claims in Question 2.