Gaussian regression toward the mean

ID: gaussian-regression-toward-the-mean

Take the original synthetic model and , mutually independent random variables, with the second argument denoting variance. The pair has a multivariate normal distribution with mean zero, unit marginal variances, and Pearson correlation coefficient . Its conditional multivariate normal distribution is
For a first reading , the second reading has conditional expectation and conditional variance , hence standard deviation . There is still conditional probability of , a movement farther from zero, where is the standard normal distribution function. The line describes a conditional mean, not an individual trajectory.
Figure 1.
An extreme first reading predicts a less extreme conditional mean, with substantial scatter
. Original synthetic scatter plot of 1,800 generated pairs, explicitly not actual measurements. Each pair has unit marginal variances and Pearson correlation coefficient ; orange points satisfy . The blue line is the conditional expectation , the band is one conditional standard deviation on either side, and the dashed line denotes . The band is not a confidence interval for an estimated line. The axes show a finite display window. Details.
A short derivation isolates . Its covariance with is zero and its variance is . Because has a multivariate normal distribution, zero covariance implies independence: independently of . This also explains why reversing the two measurements gives . The statistical association does not identify a direction of causal effect.

New to topics? Read the docs here!