= Covariance
{title2=$\operatorname{Cov}(X,Y)$}
{wiki}
The <covariance> measures the average product of two centered deviations. For real <square-integrable> <random variables> $X$ and $Y$,
$$
\operatorname{Cov}(X,Y)
=\mathbb E[(X-\mathbb EX)(Y-\mathbb EY)]
=\mathbb E[XY]-\mathbb E X\,\mathbb E Y.
$$
Deviations with the same sign contribute positively; deviations with opposite signs contribute negatively. Their magnitudes and <probabilities> determine the net result. Taking $Y=X$ recovers the <variance>. The definitions and algebraic properties are presented in https://ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-2022/mit18_05_s22_class07-prep-b.pdf[Orloff and Bloom, MIT 18.05, Covariance and Correlation, sections 2–2.1].
For a worked example, let $X,Z$ be <independent random variables>, each equally likely to be $-1$, $0$ or $1$, and put $Y=2X+Z$. All nine pairs $(X,Z)$ have <probability> $1/9$. The <expected values> are zero, while <linearity of covariance> and the vanishing <covariance> of <independent random variables> give
$$
\operatorname{Var}(X)=\operatorname{Var}(Z)=\frac23,\qquad
\operatorname{Cov}(X,Y)=2\operatorname{Var}(X)=\frac43,\qquad
\operatorname{Var}(Y)=4\operatorname{Var}(X)+\operatorname{Var}(Z)=\frac{10}{3}.
$$
The shared $X$ term makes large $X$ accompany large $Y$ on average. Replacing $2X$ by $-2X$ reverses the <covariance> to $-4/3$ without changing either <variance>.
\Image[/covariance-and-dependence.png]
{title=Positive, negative, independent, and nonlinear relationships have different covariance behavior}
{description=In the first three panels, $X$ and $Z$ independently take $-1$, $0$ and $1$ with equal <probabilities>; each point has <probability> $1/9$. In the last panel, $Y=X^2$, and the three marked outcomes each have <probability> $1/3$. The thin curve shows that <quadratic function>. Its zero <covariance> leaves strong nonlinear <statistical dependence> visible.}
The <covariance> is useful whenever deviations are added or subtracted. Expanding the square gives
$$
\operatorname{Var}(X+Y)
=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y),\qquad
\operatorname{Var}(X-Y)
=\operatorname{Var}(X)+\operatorname{Var}(Y)-2\operatorname{Cov}(X,Y).
$$
For the worked model, these are $20/3$ and $4/3$. Positive <covariance> increases variation in the sum while reducing variation in the difference: the same shared fluctuation reinforces one combination and partly cancels in the other. The identity applies beyond <independent random variables>.
The size of a <covariance> depends on the measurement scale. For constants $a,b,c,d$, <linearity of covariance> gives $\operatorname{Cov}(aX+b,cY+d)=ac\operatorname{Cov}(X,Y)$. Adding offsets has no effect; rescaling either measurement rescales the <covariance>. The <Pearson correlation coefficient> divides by both <standard deviations> to describe the standardized linear relationship.
Back to article page
