Past exam of the mathematics course of the University of Cambridge / 2002 / iii / Paper 40 / 4 / b / iii

The E-step uses the posterior class probabilitiesLet $n_j^{(m)}={\sum }_i{w}_{ij}^{(m)}$, so $n_1^{(m)}+n_2^{(m)}=n$. Up to constants, the expected complete-data log-likelihood isMaximizing the first term with ${\alpha }_1+{\alpha }_2=1$ gives ${\alpha }^{(m+1)}=n_1^{(m)}/n$. Differentiating in each ${\mu }_j$ gives ${\mu }_j^{(m+1)}={\sum }_i{w}_{ij}^{(m)}{x}_i/n_j^{(m)}$. After these updates, maximizing in the shared variance givesThe residuals use the new component means, while the weights remain from the old parameters. The divisor is $n$, not $n-2$, because this is a maximum-likelihood update. Starting with $0<\alpha <1$ and $v>0$ gives positive responsibilities and positive $n_j$; a zero effective component size at a boundary would leave its mean unidentified. These are the EM for Gaussian mixtures with a common variance transitions.