Past exam of the mathematics course of the University of Cambridge / 2009 / iii / Paper 40 / 6 / d

Both EM algorithm and data augmentation exploit a latent completion $z$, but they use it differently. EM takes a conditional expectation of the complete-data log-likelihood and then maximizes over the parameter, producing a deterministic sequence of point estimates when the M-step is uniquely specified.

In Bayesian data augmentation, choose a prior distribution $p(\theta )$ and targetAlternate the Gibbs sampler updatesThese are stochastic draws from full conditional distributions, not an expectation followed by maximization. Their invariant law is the augmented joint posterior, and discarding $z$ gives posterior inference for $\theta$, including uncertainty rather than only an MLE. A sampled completion followed by maximization would instead be a stochastic EM variant. Individual likelihood values along a data-augmentation chain need not increase: the chain is designed to sample a posterior, not to ascend the likelihood.