= Input distribution optimization for a binary symmetric channel
{title2=$I(X;Y)=h_2(p+(1-2p)q)-h_2(p)$}
Let $q=\Pr(X=1)$ for the input <bit> of a <binary symmetric channel> with crossover <probability> $p$. The <law of total probability> gives
$$
\Pr(Y=1)=(1-q)p+q(1-p)=p+(1-2p)q.
$$
Consequently its <mutual information> is
$$
I(X;Y)=h_2\!\left(p+(1-2p)q\right)-h_2(p).
$$
Here $p$ specifies the noise and $q$ specifies the input bias; they are different choices. At $p=0.1$ and $q=0.2$, the output has $\Pr(Y=1)=0.26$, giving $H(Y)\approx0.82674637$ and $I(X;Y)\approx0.35775078$ <bits>. With the same noise and the fair input $q=1/2$, the output becomes fair and $I(X;Y)\approx0.53100441$ <bits>, attaining <binary symmetric channel capacity>.
\Image[/binary-channel-input-optimization.png]
{title=A fair input maximizes retained information through a binary symmetric channel}
{description=The left panel holds the crossover <probability> fixed and varies $q=\Pr(X=1)$. The right panel shows the output <probability distributions> for $p=0.1$ with input biases $q=0.2$ and $q=0.5$. More output <Shannon entropy> helps here because the conditional noise <information entropy> stays fixed.}
For $p\ne1/2$, <binary entropy> reaches one only at output mass $1/2$, and the equation $p+(1-2p)q=1/2$ forces $q=1/2$. Thus the maximizing input is unique in those cases. For $p=1/2$, every input produces the same fair output and zero <mutual information>, so every input bias is optimal and none transmits information. A fixed input $q=0$ or $q=1$ also carries zero <mutual information> for every $p$: noise can make its output unpredictable without creating an unknown message to recover.
Back to article page
