Gibbs inequality makes mutual information nonnegative, so . The two conditional-Shannon entropy expressions giveEach Shannon entropy is bounded by the base-two logarithm of its alphabet size. Maximizing over the input distribution therefore gives for the smaller size .
Articles by others on the same topic
There are currently no matching articles.