= Solution
The <expectation-maximization algorithm> alternates conditional averaging over missing data with maximization over the parameter. Start at a parameter value $\theta^{(0)}$ for which the observed <likelihood function> is positive. At iteration $t$, its E-step forms
$$
Q(\theta\mid\theta^{(t)})=E_{\theta^{(t)}}[\log f(x,Z;\theta)\mid x],
$$
using the current conditional law of $Z$ but treating $\theta$ as the candidate parameter. Its M-step takes
$$
\boxed{\theta^{(t+1)}\in\arg\max_\theta Q(\theta\mid\theta^{(t)}).}
$$
Repeat until the parameter or observed <log-likelihood> stabilizes. An increase of $Q$ implies an increase of the observed <log-likelihood>: if $q_t(z)=f(z\mid x;\theta^{(t)})$, then
$$
\ell(\theta)-\ell(\theta^{(t)})
=Q(\theta\mid\theta^{(t)})-Q(\theta^{(t)}\mid\theta^{(t)})
+D_{\mathrm{KL}}(q_t\|f(\cdot\mid x;\theta)),
$$
where <Kullback-Leibler divergence> is nonnegative. In general this monotonicity does not guarantee a global maximum; distinct initializations may approach different stationary points. The particular observed <log-likelihood> below is concave, which makes a stronger conclusion possible there.
Back to article page
