Concavity of mutual information in the input distribution
ID: concavity-of-mutual-information-in-the-input-distribution
For a fixed finite discrete memoryless channel , the function is concave on the input probability simplex. Indeed, the induced output law is linear in , the output Shannon entropy is concave, and is linear. Their difference is therefore concave. Equivalently, at a mixture , the concavity gap is by Gibbs inequality. Thus every local maximum is a global maximum, and averaging over symmetries of the fixed channel cannot decrease mutual information.
New to topics? Read the docs here!