= Solution
For observed data $x$ and <latent variables> $Z$, let $\ell_c(\theta;x,Z)=\log p_\theta(x,Z)$ be the complete-data <log-likelihood>. From a current parameter $\theta^{(m)}$, the <expectation-maximization algorithm> computes
$$
Q(\theta\mid\theta^{(m)})=\mathbb E_{\theta^{(m)}}[\ell_c(\theta;x,Z)\mid x]
$$
and then chooses $\theta^{(m+1)}$ maximizing this function in its first argument. Thus the E-step averages the missing information conditional on the current model and observed data; the M-step treats those expectations as fixed while maximizing.
For a short proof of <likelihood function> monotonicity, let $q_m(Z)=p_{\theta^{(m)}}(Z\mid x)$. Then
$$
\frac{p_\theta(x)}{p_{\theta^{(m)}}(x)}=\mathbb E_{q_m}\frac{p_\theta(x,Z)}{p_{\theta^{(m)}}(x,Z)}.
$$
The <Jensen inequality> for the logarithm gives $\ell(\theta;x)-\ell(\theta^{(m)};x)\geq Q(\theta\mid\theta^{(m)})-Q(\theta^{(m)}\mid\theta^{(m)})$. The M-step makes the right side nonnegative. Thus observed <likelihood function> cannot decrease. This proves the usual <EM likelihood monotonicity>, while allowing convergence to local stationary points rather than guaranteeing a global maximum.
Back to article page
