Historical source for total correlation
ID: historical-source-for-total-correlation
The abstract of Information Theoretical Analysis of Multivariate Correlation by Satosi Watanabe (1960, printed p. 66) names the sum of individual information entropies minus joint entropy "total correlation." It presents this as the finest grouping of the variables and identifies applications to geometric pattern recognition and shuffling cards marked zero or one. The name therefore connects the uncertainty of a system with the structure linking its parts; the worked integer parity and noisy-integer parity models here are separate examples calculated from their stated laws.
New to topics? Read the docs here!