 Positive covariance

Two random variables have positive covariance when their centered product has positive expected value:Assume throughout that their second moments are finite. Outcomes where the centered values have the same sign contribute positively; opposite signs contribute negatively. The sign describes their average balance, rather than requiring every outcome to have matching signs.^[ref]

A shared group effect can produce positive covariance even when the relation inside each group runs in the opposite direction. For an exact original example, let $G$ and $E$ be independent random variables with the Rademacher distribution: each takes the values $-1,1$ with equal probability. Define $X=2G+E$ and $Y=2G-E$. Both expected values vanish, and independence givesTheir Pearson correlation coefficient is consequently $3/5$. The four equiprobable outcomes $(X,Y)$ are $(-3,-1)$, $(-1,-3)$, $(1,3)$ and $(3,1)$.

For either fixed value $G=g$, however, $Y=4g-X$. Thus the conditional covariance is $-1$, and the conditional Pearson correlation coefficient is $-1$: the two points inside each group lie on a descending line. The law of total covariance identifies exactly where the positive pooled value comes from:The first term averages the conditional covariances; the second measures how the two conditional expectations move together across groups. Here the between-group contribution outweighs the negative within-group contribution. This is a covariance form of Simpson's paradox.

The dashed linear regression line has slope $\mathrm{Cov}(X,Y)/\mathrm{Var}(X)=3/5$, but that slope does not give the direction of the relation at fixed $G$. Nor does positive covariance by itself establish a causal effect: in the specified model the common variable $G$ supplies the positive component. Choosing which variables to condition on is a question about the model and the intended comparison, beyond computing one number.

Positive covariance is also weaker than requiring one variable to be an increasing function of the other. Under appropriate integrability assumptions, strict positive covariance with an increasing payoff supplies a useful sufficient condition; the grouping example shows a different route to the same sign.