= Two-Gaussian mixture example
{c}
{title2=$X\sim\tfrac13N(-2,1)+\tfrac23N(2,1)$}
Stipulate an original <Gaussian mixture distribution> with <mixture weights> $\pi_1=1/3$, $\pi_2=2/3$, component <means> $\mu_1=-2$, $\mu_2=2$, and component <variances> both equal to one. These are chosen <statistical parameters>, with no observed sample or fitting. If $\phi(t)=(2\pi)^{-1/2}e^{-t^2/2}$ is the <standard normal distribution> <probability density function>, then
$$
f(x)=\frac13\phi(x+2)+\frac23\phi(x-2).
$$
In the figure, the two colored curves are weighted <probability density functions>, with areas $1/3$ and $2/3$. Adding their heights at every $x$ gives the black curve. Neither colored curve stops at a dividing line.
\Image[/two-gaussian-mixture-density-decomposition.png]
{title=Stipulated two-Gaussian mixture: weighted component densities and their sum}
{description=Original analytic curves for component means $-2$ and $2$, component variances one, and mixture weights $1/3$ and $2/3$. No observed points, histogram, or fitted parameters are used.}
The <law of total expectation> and <law of total variance> give the total <mean> and <variance>:
$$
\mathbb E[X]=\frac13(-2)+\frac23(2)=\frac23,
\qquad
\operatorname{Var}(X)=\underbrace{1}_{\mathbb E[\operatorname{Var}(X\mid Z)]}+\underbrace{\frac13\left(-2-\frac23\right)^2+\frac23\left(2-\frac23\right)^2}_{\operatorname{Var}(\mathbb E[X\mid Z])=32/9}=\frac{41}{9}.
$$
All of the extra <variance> $32/9$ comes from variation between component <means>. Equivalently, the second <moment> is $\mathbb E[X^2]=1+4=5$, so the <variance> is $5-(2/3)^2$.
A <mixture distribution> is different from a deterministic <linear combination> of <random variables>. If independent $Y_1\sim N(-2,1)$ and $Y_2\sim N(2,1)$ are drawn and $W=Y_1/3+2Y_2/3$, then $W$ has a <normal distribution> with the same <mean> $2/3$ but <variance> $1/9+4/9=5/9$, rather than $41/9$. Every draw of $W$ uses both <random variables>. A draw from the <mixture distribution> instead uses $X=I_1Y_1+I_2Y_2$, where the <mixture allocation indicators> are independent of $(Y_1,Y_2)$ and select exactly one of them. The expression for $f$ adds densities, not simultaneous random outcomes.
Back to article page
