= Noisy parity total correlation
{title2=$\operatorname{TC}(X,Y,Z)=1-h_2(\varepsilon)$}
Keep $X,Y$ as <independent> fair <bits>, introduce an <independent> <Bernoulli random variable> $N$ with $\mathbb P(N=1)=\varepsilon$, and set $Z=X\mathbin\oplus Y\mathbin\oplus N$. The extra <bit> flips the <integer parity> rule with <probability> $\varepsilon$. For each even-<integer parity> triple the <probability> is $(1-\varepsilon)/4$; for each odd-<integer parity> triple it is $\varepsilon/4$.
Every coordinate remains fair, and every pair remains <independent> for every $0\leq\varepsilon\leq1$. For example, after fixing $X$, the fair independent $Y$ makes $Y\mathbin\oplus N$ fair regardless of $\varepsilon$. A pair's <joint probability distribution> therefore does not change as noise destroys the three-way relation.
\Image[/total-correlation-noisy-parity-distributions.png]
{title=Noise changes the joint parity distribution without changing any pair's independence}
{description=Calculated <probability mass functions> for the stipulated three-<bit> model at $\varepsilon=0,1/4,1/2$. Even-<integer parity> and odd-<integer parity> triples have different colors. The dashed line is the product of the three fair marginal <probability distributions>. These are theoretical <probabilities>, not measurements.}
Knowing $X,Y$, uncertainty in $Z$ is exactly the uncertainty in $N$. The <chain rule for information entropy> gives
$$
H(X,Y,Z)=H(X,Y)+H(Z\mid X,Y)=2+h_2(\varepsilon),
$$
where $h_2$ is <binary entropy>. Hence
$$
\boxed{\operatorname{TC}(X,Y,Z)=1-h_2(\varepsilon).}
$$
At $\varepsilon=1/4$, the result is approximately $0.188722$ <bits>, although all three pairwise <mutual informations> are zero. At $\varepsilon=1/2$, all eight triples have <probability> $1/8$, giving joint <independence> and zero <total correlation>. At $\varepsilon=1$, the complementary <integer parity> rule is deterministic again, so <total correlation> returns to one <bit>.
\Image[/total-correlation-parity-noise-and-pairwise-information.png]
{title=Total correlation detects a noisy three-way constraint missed by pairwise mutual information}
{description=The exact formula $1-h_2(\varepsilon)$ for the stipulated <noisy parity total correlation>, plotted over $0\leq\varepsilon\leq1/2$. The <mutual information> of each pair is identically zero; joint <independence> occurs only at the right endpoint of this interval.}
The example distinguishes "there is a dependence" from "a pair reveals it." Once $X,Y$ are known together, $Z$ is predictable at zero noise; knowing either one alone leaves a fair <bit>. <Total correlation> detects the joint departure from <independence>, but does not by itself specify which variables cause which others.
Back to article page
