Past exam of the mathematics course of the University of Cambridge / 2009 / iii / Paper 40 / 6 / b

Write ${\theta }_0={\theta }^{(t)}$ and $h_0(z)=f(x,z;{\theta }_0)/L(x\mid {\theta }_0)$. Assume the likelihoods are positive and the expected log likelihoods used in the step are finite. On the support where $h_0>0$, let $r(z)=f(x,z;\theta )/f(x,z;{\theta }_0)$. Integrating over that support givesIt is an equality when the candidate joint density has no additional mass outside the old support. Concavity of the logarithm and the Jensen inequality now implyThus the argument does not silently require an identical support at every parameter. The M-step selects a value whose right-hand side is nonnegative, proving EM likelihood monotonicity:When the conditional densities allow finite divergence, the same calculation has the informative exact formThe Kullback-Leibler divergence is nonnegative, since ${\mathbb{E}}_{h_0}\log (h_{\theta }/h_0)\le \log {\int }_{\{h_0>0\}}{h}_{\theta }\le 0$. This also explains why an increase of the surrogate objective guarantees an increase of the observed objective, while equality of the two likelihoods is possible.