= Chain decomposition of total correlation
{title2=$\operatorname{TC}(X_1,\ldots,X_n)=\sum_{i=2}^n I(X_i;X_1,\ldots,X_{i-1})$}
For finite <alphabets>, expanding <joint entropy> with the <chain rule for information entropy> gives
$$
\operatorname{TC}(X_1,\ldots,X_n)
=\sum_{i=2}^n\bigl[H(X_i)-H(X_i\mid X_1,\ldots,X_{i-1})\bigr]
=\sum_{i=2}^n I(X_i;X_1,\ldots,X_{i-1}).
$$
Each term is <mutual information> with an entire preceding block, rather than necessarily with one preceding variable. Changing the order changes these individual terms but leaves their sum unchanged.
For the noiseless <parity constraint and total correlation>, $I(Y;X)=0$ and $I(Z;X,Y)=1$ <bit>. The conditional block term supplies exactly the dependence that the pairwise comparisons missed. For three identical copies of a fair <bit>, the two block terms are both one <bit>. Thus <total correlation> counts total departure from the product law, not merely the presence of a single irreducible multiway relation.
Back to article page
