= Solution
Both <data augmentation> and the <expectation-maximization algorithm> introduce <latent variables> $Z$ that make the complete-data model easier to handle. The observed-data likelihood is obtained by summing or integrating $p(x,z\mid\theta)$ over the missing data, and both methods use the conditional law of $Z$ given the observations and current parameters.
In <data augmentation> for Bayesian inference, one alternates random draws
$$
Z^{(r+1)}\sim p(z\mid x,\theta^{(r)}),\qquad\theta^{(r+1)}\sim p(\theta\mid x,Z^{(r+1)}).
$$
These are <Gibbs sampler> updates on the joint <posterior distribution>; marginal parameter draws explore the whole posterior, including uncertainty, provided the chain converges appropriately. The parameter conditional incorporates its prior.
The <expectation-maximization algorithm> is instead a deterministic optimization procedure. Its E step forms
$$
Q(\theta\mid\theta^{(r)})=\mathbb E_{Z\mid x,\theta^{(r)}}[\log p(x,Z\mid\theta)],
$$
and its M step maximizes this function over $\theta$. Exact steps do not decrease the observed-data <likelihood function>. Their aim is a likelihood stationary point, often a local maximum, rather than draws from a parameter distribution. Adding a log prior to the maximization gives a posterior-mode version, which still does not sample the posterior.
\b[Data augmentation samples conditional missing data and parameters; EM averages the complete-data log likelihood and optimizes parameters.] The E step is an expectation of the log likelihood, not the log of an averaged likelihood, and generally is not equivalent to merely replacing each missing value by its conditional mean. A Monte Carlo E-step approximation is a further variant of EM, not automatically the same procedure as a <data augmentation> chain.
Back to article page
