Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 41 / 5 / Solution

Let $X$ denote the observed data, $Z$ the unobserved data and $\theta$ the model parameters. The expectation-maximization algorithm works with the complete-data log-likelihood ${\ell }_c(\theta ;X,Z)$. Given ${\theta }^{(r)}$, its E-step computeswhere the conditional expectation uses the old parameter, while $\theta$ remains the argument to maximize. Its M-step chooses ${\theta }^{(r+1)}$ maximizing this function over the parameter space. EM likelihood monotonicity ensures that an M-step increasing $Q$ cannot decrease the observed-data log-likelihood. It does not guarantee a global maximum, so convergence and any parameter constraints must still be checked.