= Solution
Fix the channel matrix $W$. The mixed input induces the mixed output $q_3=\lambda q_1+(1-\lambda)q_2$. The <concavity of information entropy> implies
$$
H(q_3)\geq\lambda H(q_1)+(1-\lambda)H(q_2).
$$
For completeness, the difference is $\lambda D(q_1\Vert q_3)+(1-\lambda)D(q_2\Vert q_3)\geq0$ by <Gibbs inequality>, including zero coordinates with the usual conventions.
On the other hand, conditional output <Shannon entropy>[entropy] is linear in the input law:
$$
H(Y_3\mid X_3)=\sum_xp_3(x)H(W(\cdot\mid x))
=\lambda H(Y_1\mid X_1)+(1-\lambda)H(Y_2\mid X_2).
$$
Subtracting gives the <concavity of mutual information in the input distribution>:
$$
\boxed{I(X_3;Y_3)\geq\lambda I(X_1;Y_1)+(1-\lambda)I(X_2;Y_2).}
$$
This makes maximizing <mutual information> over the input <probability simplex> a concave optimization problem: a local maximum is global. It also justifies averaging input distributions over channel symmetries when searching for an optimum, since symmetry averaging cannot lower the achieved <mutual information>. The channel itself is held fixed throughout.
Back to article page
