Conditional variance ($\mathrm{Var}(X\mid \mathcal{G})$)

The conditional variance measures the remaining squared uncertainty after specified information has been used to form a conditional mean. For a real square-integrable random variable $X$ and a sub-sigma-algebra $\mathcal{G}$, let $m=\mathbb{E}[X\mid \mathcal{G}]$. ThenIt is a $\mathcal{G}$-measurable random variable, nonnegative almost surely, rather than necessarily a single number. With the information $\mathcal{G}=\sigma (Y)$, its value is the variance in the group identified by $Y$. Its expected value is the mean squared error of predicting $X$ by its conditional mean.^[ref]

In the exact mixture distribution in law of total expectation, group $A$ contains $0,4$ around mean two, and group $B$ contains $6,10$ around mean eight. Both groups have conditional variance four because their deviations from their own means are $-2,2$ with equal probabilities. Therefore $\mathbb{E}[\mathrm{Var}(X\mid Y)]=4$. Meanwhile, the two group means have variance $\frac{1}{4}(2-13/2{)}^2+\frac{3}{4}(8-13/2{)}^2=27/4$. The law of total variance givesDirect calculation gives $\mathbb{E}[X^2]=53$ and $\mathrm{Var}(X)=53-(13/2{)}^2=43/4$, checking the decomposition.

For any square-integrable $\mathcal{G}$-measurable predictor $Z$, the error identity isThe cross term vanishes because $\mathbb{E}[X-m\mid \mathcal{G}]=0$. Thus using the conditional mean minimizes mean squared error, while conditional variance quantifies what this information still cannot predict. This is the orthogonal projection interpretation in L2 space.^[ref]

For square-integrable $X$ and nested information $\mathcal{H}\subseteq \mathcal{G}$, let $m_{\mathcal{H}}=\mathbb{E}[X\mid \mathcal{H}]$ and $m_{\mathcal{G}}=\mathbb{E}[X\mid \mathcal{G}]$. The predictor $m_{\mathcal{H}}$ is also $\mathcal{G}$-measurable, so the conditional variance error identity givesMore information cannot increase the average best squared prediction error. Equality holds exactly when the two conditional means agree almost surely; additional information can change the conditional distribution without changing its mean.

This is an averaged statement. Take $X=-4,0,4$ with equal probabilities, and learn only whether $X$ is zero. The unconditional variance is $32/3$. In the nonzero group, the conditional variance is $16$, which is larger; in the zero group it is zero. Their weighted average remains $(2/3)16+(1/3)0=32/3$. Both groups have conditional mean zero, so the extra information has not improved squared-error prediction, even though it has revealed whether the outcome is extreme or central. An individual group's conditional variance need not decrease.