= Removing a redundant input symbol from a channel
Suppose one row of a finite <discrete memoryless channel> is a <convex combination> of other rows: $W(\cdot\mid x_0)=\sum_j\alpha_jW(\cdot\mid x_j)$. Deleting input $x_0$ leaves the <channel capacity> unchanged. Given any input law with mass $t$ on $x_0$, move that mass to the other symbols in proportions $\alpha_j$. The output law stays the same. By the <concavity of information entropy>, $H(W(\cdot\mid x_0))\geq\sum_j\alpha_jH(W(\cdot\mid x_j))$, so this replacement cannot increase <conditional entropy> $H(Y\mid X)$ and cannot decrease <mutual information>. Taking maxima proves the claim; the smaller channel is also trivially a restriction of the original one. If the entropy inequality is strict, every positive use of the redundant row is strictly suboptimal compared with its redistribution.
Back to article page
