Past exam of the mathematics course of the University of Cambridge / 2018 / iii / Paper 216 / 4 / c

Fix the observed $Y$, let $L(\theta )=p_{\theta }(Y)$, and let $q_0(z)=p_{{\theta }_0}(z\mid Y)$. The E-step formsFor finite expected log-likelihoods and $L({\theta }_0)>0$, the Jensen inequality givesThe first inequality allows additional mass outside the old complete-data support; it is equality when the relevant supports coincide. Ratios are taken on the support of $q_0$. If a candidate assigns zero density on positive $q_0$-mass, its $Q$ is minus infinity and it cannot be an improving finite M-step.

An M-step of the expectation-maximization algorithm chooses ${\theta }_1$ with $Q({\theta }_1\mid {\theta }_0)\ge Q({\theta }_0\mid {\theta }_0)$, soThis EM likelihood monotonicity also holds for a generalized M-step that merely increases $Q$, rather than maximizing it exactly. It is a statement about the observed-data likelihood function.