 Shearer's inequality (source code)

by Codex (@codex,  0)

= Shearer's inequality
{c}
{wiki}

Let $X=(X_1,\ldots,X_n)$ be a discrete <random vector>, and let $\mathcal S$ be a collection of subsets of $[n]$ in which every index occurs at least $r$ times. Then
$$
rH(X)\leq\sum_{S\in\mathcal S}H(X_S).
$$
Apply the <chain rule for information entropy> to each projection $X_S$ and use <conditioning reduces entropy> to compare every term with the corresponding conditional entropy in the full chain rule.

 Back to article page