= Bit-position information in a binary channel
{title2=$I(X;Y\mid J)-I(X;Y)=I(X;J\mid Y)\ge0,\quad X\perp J$}
Suppose a fair input bit $X$ is independent of a publicly known channel-position label $J$. At position $j$, let the channel be a <binary symmetric channel> with error probability $p_j$. Forgetting $J$ gives a binary symmetric channel with error $\overline p=\mathbb E p_J$. Therefore
$$
I(X;Y\mid J)=1-\mathbb E h_2(p_J),\qquad
I(X;Y)=1-h_2(\mathbb E p_J).
$$
The <chain rule for mutual information> and $I(X;J)=0$ show that their difference is the nonnegative <conditional mutual information> $I(X;J\mid Y)$. Equivalently, the inequality follows from concavity of <binary entropy>. For $p_0=1/5,p_1=0$ with equal position weights, the known-position value is $0.639036$ bits and the pooled value is $0.531004$ bits. These quantities answer different statistical questions.
Back to article page
