= Concavity of mutual information in the input distribution
For a fixed finite <discrete memoryless channel> $W$, the function $p\mapsto I_p(X;Y)$ is concave on the input probability simplex. Indeed, the induced output law $q=pW$ is linear in $p$, the output <Shannon entropy> $H(q)$ is concave, and $H(Y\mid X)=\sum_xp(x)H(W(\cdot\mid x))$ is linear. Their difference is therefore concave. Equivalently, at a mixture $p=\lambda p_1+(1-\lambda)p_2$, the concavity gap is $\lambda D(q_1\Vert q)+(1-\lambda)D(q_2\Vert q)\geq0$ by <Gibbs inequality>. Thus every local maximum is a global maximum, and averaging over symmetries of the fixed channel cannot decrease <mutual information>.
Back to article page
