Past exam of the mathematics course of the University of Cambridge / 2009 / iii / Paper 40 / 6 / a / Solution

The observed-data likelihood function integrates out the latent variable:Choose an initial parameter ${\theta }^{(0)}$ with positive likelihood. At iteration $t$, the E-step of the EM algorithm forms the entire expected complete-data log-likelihoodThe expectation is under the conditional missing-data law at the old parameter, while the argument $\theta$ remains free. The M-step then choosesThe observed data $x$ stay fixed throughout. This is not obtained merely by substituting the conditional mean of $z$ into a nonlinear log-likelihood: the expectation of all required complete-data terms must be taken. A generalized EM step only requires $Q({\theta }^{(t+1)}\mid {\theta }^{(t)})\ge Q({\theta }^{(t)}\mid {\theta }^{(t)})$, which suffices for the monotonicity proved next. Exact maximization assumes that an admissible maximizer exists.