Past exam of the mathematics course of the University of Cambridge / 2008 / iii / Paper 47 / 6 / a / Solution

The expectation-maximization algorithm alternates conditional averaging over missing data with maximization over the parameter. Start at a parameter value ${\theta }^{(0)}$ for which the observed likelihood function is positive. At iteration $t$, its E-step formsusing the current conditional law of $Z$ but treating $\theta$ as the candidate parameter. Its M-step takesRepeat until the parameter or observed log-likelihood stabilizes. An increase of $Q$ implies an increase of the observed log-likelihood: if $q_t(z)=f(z\mid x;{\theta }^{(t)})$, thenwhere Kullback-Leibler divergence is nonnegative. In general this monotonicity does not guarantee a global maximum; distinct initializations may approach different stationary points. The particular observed log-likelihood below is concave, which makes a stronger conclusion possible there.