= Parity constraint and total correlation
{title2=$Z=X\mathbin\oplus Y,\quad\operatorname{TC}(X,Y,Z)=1\text{ bit}$}
Let $X,Y$ be <independent> fair <bits> and set $Z=X\mathbin\oplus Y$, their <exclusive or>. The four possible triples are $000,011,101,110$, each with <probability> $1/4$. Every coordinate is fair, and every pair has all four outcomes with equal <probability>. Thus each pair's <mutual information> is zero, as in <pairwise independence does not imply joint independence>.
The full <random vector> has <joint entropy> $\log_2 4=2$ <bits>, while the marginal <information entropies> sum to three <bits>. Consequently
$$
\boxed{\operatorname{TC}(X,Y,Z)=3-2=1\text{ bit}.}
$$
The product of the marginals predicts <probability> $1/8$ for every triple, including the four impossible ones. On each possible triple, the ratio of true to product <probability> is $(1/4)/(1/8)=2$; averaging its base-two <logarithm> gives the same one <bit> of <relative entropy>.
The example gives a concrete use for joint information measures. A <lossless source coding> scheme that treats the three coordinates separately needs three <bits> here. If the <decoder> knows the <integer parity> rule, the <encoder> can send just $X,Y$ and let the <decoder> reconstruct $Z$ exactly. Pairwise <independence> does not reveal that saving. This is a stipulated source law and an explicit two-<bit> code, rather than a promise that arbitrary data can be shortened.
Back to article page
