Past exam of the communication Theory course of the University of Oxford / 2011 / 3 / b / ii

There are three distinguishable output groups: columns $\{1,2\}$ can arise only from input one; column $\{3\}$ can arise from inputs two or three, whose rows are identical; and columns $\{4,5\}$ can arise only from input four.

Each group has zero internal channel capacity. A block with one input cannot carry a variable input choice, and the middle block has identical output laws for both of its inputs. Combining the three disjoint zero-capacity blocks therefore givesEquivalently, the output identifies exactly which of these three groups was used, so mutual information is the entropy of that group label. It is maximized when all group probabilities are one third. The capacity-achieving input distributions are preciselySplitting the middle probability arbitrarily cannot communicate an extra distinction because its two channel rows agree.