Past exam of the mathematics course of the University of Cambridge / 2005 / iii / Paper 37 / 4 / a / Solution

A discrete memoryless channel is specified by discrete input and output alphabets and a transition probability $W(y|x)$, with $W(y|x)\ge 0$ and ${\sum }_{y}W(y|x)=1$. Its successive outputs are conditionally independent given the inputs, with the same transition rule at every use:Operationally, its channel capacity is the supremum of communication rates in bits per use attainable by increasingly long block codes with decoding error probability tending to zero. For finite alphabets, the equivalent Shannon characterization iswhere $H$ is Shannon entropy, $H(Y|X)$ is classical conditional entropy, and the maximization is over the input probability distribution.

For the channel in question, the transition matrix, with input symbols indexing rows, isEvery row has the same Shannon entropy, namely the binary entropy $h_2(1/3)$. Consequently $H(Y|X)=h_2(1/3)$ for every input probability distribution. Since the output has three symbols, the maximum entropy on a finite alphabet givesChoose a uniform input. Each column of $W$ sums to one, so $P_Y(y)=1/3$ for every output symbol and the upper bound is attained. FinallyThus the cyclic ternary channel capacity isachieved by a uniform input probability distribution. This is a weakly symmetric channel capacity calculation: the matrix rows are permutations and the column sums agree. The channel is not the usual ternary symmetric-error channel, since an error has only one possible changed output for each input.