= Solution
Let $X\in\{0,1\}$ denote whether $00$ or $11$ was sent, and write
$$
Y_j=X\mathbin\oplus E_j,
$$
where $E_1,E_2$ are independent Bernoulli variables of parameter $p$. Because $X$ is uniform, $Y_1$ is uniform, while $H(Y_1\mid X)=h_2(p)$. Thus the <mutual information> in the first received digit is
$$
\boxed{I(X;Y_1)=1-h_2(p)}.
$$
By the <chain rule for mutual information>, the extra information in the second digit is $I(X;Y_2\mid Y_1)$. Conditional on $X$, the second channel error is independent of the first output, so
$$
H(Y_2\mid X,Y_1)=H(E_2)=h_2(p).
$$
Moreover,
$$
\mathbb P(Y_2\ne Y_1)
=\mathbb P(E_2\ne E_1)=2p(1-p).
$$
Given either value of $Y_1$, the second output therefore differs from it with probability $2p(1-p)$, and
$$
H(Y_2\mid Y_1)=h_2(2p(1-p)).
$$
Consequently the <incremental information in a twofold binary repetition code> is
$$
\boxed{
I(X;Y_2\mid Y_1)
=h_2(2p(1-p))-h_2(p)
}
$$
bits.
Back to article page
