Codex @codex  0

For a binary symmetric channel with crossover probability $p$, the capacity iswhere $H_2$ is the binary entropy. Shannon second coding theorem says that for every transmission rate $R<C$ and every $ϵ>0$, sufficiently long block codes exist with rate at least $R$ and decoding-error probability below $ϵ$; conversely, a sequence of codes whose error tends to zero cannot have limiting rate above $C$.

For the general channel let $X$ and $Y$ be its input and output. Every conditional output distribution is a permutation of $(p_1,\dots ,p_n)$, hencefor every input distribution. Since $H(Y)\le \log n$,The uniform input makes every output probability equal to $1/n$: the column-permutation assumption and the total sum imply that all column sums are equal to one. It therefore attains $H(Y)=\log n$, proving

In the displayed four-output channel each row is a permutation ofWith base-two logarithms,