Chain decomposition of total correlation ($\mathrm{TC}(X_1,\dots ,X_n)={\sum }_{i=2}^{n}I(X_i;X_1,\dots ,X_{i-1})$)

For finite alphabets, expanding joint entropy with the chain rule for information entropy givesEach term is mutual information with an entire preceding block, rather than necessarily with one preceding variable. Changing the order changes these individual terms but leaves their sum unchanged.

For the noiseless parity constraint and total correlation, $I(Y;X)=0$ and $I(Z;X,Y)=1$ bit. The conditional block term supplies exactly the dependence that the pairwise comparisons missed. For three identical copies of a fair bit, the two block terms are both one bit. Thus total correlation counts total departure from the product law, not merely the presence of a single irreducible multiway relation.