Solution
ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2009/iii/paper-40/6/b/solution
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 40 6 b Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-14
Write and . Assume the likelihoods are positive and the expected log likelihoods used in the step are finite. On the support where , let . Integrating over that support givesIt is an equality when the candidate joint density has no additional mass outside the old support. Concavity of the logarithm and the Jensen inequality now implyThus the argument does not silently require an identical support at every parameter. The M-step selects a value whose right-hand side is nonnegative, proving EM likelihood monotonicity:When the conditional densities allow finite divergence, the same calculation has the informative exact formThe Kullback-Leibler divergence is nonnegative, since . This also explains why an increase of the surrogate objective guarantees an increase of the observed objective, while equality of the two likelihoods is possible.
New to topics? Read the docs here!