= Law of total variance
{title2=$\operatorname{Var}(X)=\mathbb E[\operatorname{Var}(X\mid Y)]+\operatorname{Var}(\mathbb E[X\mid Y])$}
{wiki}
The <law of total variance> separates the spread of a population into spread inside groups and spread between their <means>. For a real <square-integrable> <random variable> $X$, so that $\mathbb E[X^2]<\infty$, and another <random variable> $Y$ on the same <probability space>,
$$
\boxed{\operatorname{Var}(X)=\mathbb E[\operatorname{Var}(X\mid Y)]+\operatorname{Var}(\mathbb E[X\mid Y]).}
$$
The first term averages the <conditional variance> over the possible information $Y$; the second measures how much the <conditional mean> changes with that information. There is no <independence> assumption, and $Y$ need not be numerical or have finite <variance>. Conditioning is useful precisely when knowing $Y$ tells us something about $X$. See https://ocw.mit.edu/courses/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/2aae0a59cd484849fec4aa12a6f18969_MIT6_041SCF13_Law_of_Total_Variance_300k.pdf[MIT tutorial on using conditional expectation and variance].
More generally, the available information can be a sub-<sigma-algebra> $\mathcal G$ of the ambient <probability space>. With $m=\mathbb E[X\mid\mathcal G]$, the existing <conditional expectation> definition gives
$$
\operatorname{Var}(X\mid\mathcal G)=\mathbb E[(X-m)^2\mid\mathcal G],\qquad
\operatorname{Var}(X)=\mathbb E[\operatorname{Var}(X\mid\mathcal G)]+\operatorname{Var}(m).
$$
Taking $\mathcal G=\sigma(Y)$ recovers the displayed formula. The <conditional mean> and <conditional variance> are themselves <random variables>: before learning the group, we do not yet know which group average or group spread will apply. Their outer <expectation> and <variance> turn these varying quantities into numbers.
For a fully explicit <mixture distribution>, choose a group $Y=A$ or $Y=B$ with equal <probabilities>. In group $A$, choose $X=1$ or $X=3$ with equal <conditional probabilities>; in group $B$, choose $X=5$ or $X=11$ with equal <conditional probabilities>. Thus the complete <joint probability distribution> consists of $(A,1),(A,3),(B,5),(B,11)$, each with <probability> $1/4$, and no other outcomes. The <conditional means> are $2$ and $8$. Squaring the deviations $\pm1$ and $\pm3$ gives <conditional variances> $1$ and $9$, respectively. The <law of total expectation> gives the overall <mean> $\mu=(2+8)/2=5$.
The within-group contribution to <variance> is
$$
\mathbb E[\operatorname{Var}(X\mid Y)]=\tfrac12(1)+\tfrac12(9)=5.
$$
For the between-group contribution, the <conditional mean> takes values $2$ and $8$, each three units from the overall <mean>. Consequently,
$$
\operatorname{Var}(\mathbb E[X\mid Y])=\tfrac12(2-5)^2+\tfrac12(8-5)^2=9,
\qquad \operatorname{Var}(X)=5+9=14.
$$
We can check this without any conditioning: the <discrete random variable> $X$ takes values $1,3,5,11$ with equal <probabilities>, so
$$
\operatorname{Var}(X)=\frac{(1-5)^2+(3-5)^2+(5-5)^2+(11-5)^2}{4}
=\frac{16+4+0+36}{4}=14.
$$
Equivalently, its <second moment> is $(1+9+25+121)/4=39$, and subtracting the squared <mean> gives $39-25=14$. These are population <variances>, with the stated <probabilities> as weights, rather than estimates using the corrected denominator of a <sample variance>.
\Image[/total-variance-mixture.png]
{title=Four equally likely outcomes split total variance into within-group 5 and between-group 9}
The figure makes the two sources of spread visible. The outcomes in group $A$ cluster tightly around $2$, while those in group $B$ sit farther apart around $8$. Replacing every outcome by its <conditional mean> removes the internal spread but leaves the separation between groups. Conversely, subtracting each group's <conditional mean> aligns the two groups around zero. Averaging only the <conditional variances> would miss nine units of <variance>; retaining only the <conditional means> would miss five. Neither summary alone describes the whole population.
Two modifications clarify what each contribution measures. Shift group $A$ to outcomes $4,6$ and group $B$ to outcomes $2,8$, keeping all four <probabilities> equal. Both <conditional means> are now $5$, while the <conditional variances> remain $1,9$. The between-group contribution vanishes, and total <variance> is just $5$. Group membership still tells us how uncertain an outcome is, even though the best point prediction stays the same. Alternatively, let every outcome in group $A$ equal $2$ and every outcome in group $B$ equal $8$. Now both <conditional variances> vanish, and all nine units of total <variance> come from group separation. Knowing the group predicts the outcome perfectly. These limiting cases distinguish differences in average outcome from differences in internal spread; the original <mixture distribution> contains both.
The same calculation answers a prediction question. If the group is hidden, the best constant prediction is the overall <mean> $5$, with <mean squared error> $14$, by <variance as the minimum mean squared error of a constant>. If the group is revealed, predict $2$ for $A$ and $8$ for $B$. The errors become $-1,1,-3,3$, and their <mean squared error> is $5$. Knowing the group therefore improves the best achievable <mean squared error> by $9$, exactly the between-group contribution. This is an interpretation relative to the information available: the remaining uncertainty could shrink further if additional information were revealed.
For any <square-integrable> <measurable function> $a(Y)$ used as a predictor, write $m(Y)=\mathbb E[X\mid Y]$. The prediction identity is
$$
\mathbb E[(X-a(Y))^2]=\mathbb E[\operatorname{Var}(X\mid Y)]+\mathbb E[(m(Y)-a(Y))^2].
$$
Thus predicting with the <regression function> $m(y)=\mathbb E[X\mid Y=y]$ gives the <conditional expectation> $m(Y)$ and minimizes <mean squared error> over every permitted predictor, including nonlinear ones. This minimizes average squared error, not the error for each individual outcome. See https://www.stat.berkeley.edu/~aditya/resources/AllLectures2018Fall201A.pdf#page=71[Berkeley Statistics 201A notes, sections 36–37].
Why do the two pieces add cleanly? Put $R=X-m(Y)$. Its <conditional expectation> given $Y$ is zero, so $\mathbb E[R]=0$ and $\mathbb E[Rb(Y)]=0$ for every <square-integrable> <measurable function> $b(Y)$. In particular, $R$ and $m(Y)$ are <uncorrelated random variables>. Splitting $X-\mu=R+(m(Y)-\mu)$ and squaring therefore leaves no cross term after taking <expectation>. Geometrically, the <conditional expectation> is an <orthogonal projection> in the <L2 space>, and the <law of total variance> is its <Pythagorean theorem> for centered <random variables>.
\Image[/conditional-mean-residual.png]
{title=Conditional prediction leaves zero group means but different residual spreads}
In our example, the four products $R(m(Y)-5)$ are $3,-3,-9,9$, which average to zero. Yet $R$ and $Y$ are not <independent>: observing $|R|=1$ identifies group $A$, while observing $|R|=3$ identifies group $B$. This is a concrete instance of <heteroscedasticity>, with different <conditional variances> despite zero <conditional means> of the errors. A group can predict uncertainty even when it cannot predict the error's sign.
The <law of total variance> is useful whenever aggregated data conceal distinct groups. Changing <mixture weights> changes both contributions, so the weighting must match the population being described. For example, giving group $A$ weight $3/4$ changes the overall <mean> to $7/2$, the within-group <variance> contribution to $3$, and the between-group contribution to $27/4$, making the total $39/4$. Group differences account for spread and prediction gains; assigning them a causal explanation requires a separate <causal inference> argument.
Back to article page
