= EM maximization step
{c}
{title2=$\theta^{(t+1)}\in\operatorname*{arg\,max}_{\theta}Q(\theta\mid\theta^{(t)})$}
= M-step
{c}
{synonym}
Maximize $Q(\theta\mid\theta^{(t)})$ over the permitted <statistical parameters>, with the <E-step> <conditional distribution> held fixed.<Maximum Likelihood from Incomplete Data via the EM Algorithm by Arthur Pentland Dempster and others>[Section 2, printed p. 6: general maximization step]{ref} A <finite mixture model> has
$$
Q(\theta\mid\theta^{(t)})=\sum_{i,j}r_{ij}^{(t)}\bigl(\log\pi_j+\log f_j(y_i;\theta)\bigr).
$$
For <normal distributions> with known common <variance> $\sigma^2>0$ and fixed positive <mixture weights>, only the <means> are fitted. Dropping terms independent of these candidate <means> gives
$$
Q(\mu\mid\theta^{(t)})=C-\frac{1}{2\sigma^2}\sum_{i,j}r_{ij}^{(t)}(y_i-\mu_j)^2,\qquad N_j^{(t)}=\sum_ir_{ij}^{(t)},\qquad \mu_j^{(t+1)}=\frac{\sum_ir_{ij}^{(t)}y_i}{N_j^{(t)}}.
$$
The <derivative> in $\mu_j$ is $\sum_ir_{ij}^{(t)}(y_i-\mu_j)/\sigma^2$, and its second <derivative> is $-N_j^{(t)}/\sigma^2$. Thus a positive weighted count gives a unique maximizing <weighted mean>. This is a maximization of the surrogate $Q$; the observed-data <log-likelihood> need not be <concave>. Recompute the <mixture responsibilities> after updating the <means>. The broader <EM for Gaussian mixtures with a common variance> also estimates <mixture weights> and the shared <variance>.
Back to article page
