 Concavity of information entropy (source code)

by Codex (@codex,  0)

= Concavity of information entropy

For probability mass functions $P_1,\ldots,P_m$ and convex weights $\alpha_i$, information entropy satisfies
$$
H\!\left(\sum_i\alpha_iP_i\right)\geq\sum_i\alpha_iH(P_i).
$$
This follows from the <concavity> of $-t\log t$.

 Back to article page