Past exam of the mathematics course of the University of Cambridge / 2019 / iii / Paper 216 / 4

Let $\theta =(\beta ,\pi )$ and regard $X_U$ as the latent variable. At iteration $t$, the E-step formsThe M-step updatesThis is the expectation-maximization algorithm for the posterior objective: including $\log p(\theta )$ in the complete-data log density makes the maximizer a maximum a posteriori estimate rather than a maximum-likelihood estimate.

Let $U_i=\{j:(i,j)\notin O\}$ and let $x_{i,O}$ denote the observed entries in row $i$. Conditional on the parameters, different rows of the missing design are independent, while the missing entries within one row are coupled by its Gaussian response. For $z\in \{0,1{\}}^{U_i}$,Normalizing this expression over the $2^{|U_i|}$ configurations and multiplying over rows gives the full conditional distribution of $X_U$.

Ignoring constants, the complete-data log posterior isTake conditional expectations under $({\beta }^{(t)},{\pi }^{(t)})$. DefineCompleting the square givesFor $v\ne 0$,so $A$ is positive definite. Exact rowwise expectations require summing over $2^{|U_i|}$ states. Thus the cost is exponential in the largest number of missing covariates in one row, more precisely ${\sum }_i{2}^{|U_i|}$ times a polynomial factor for accumulating first and second moments.

Positive definiteness makes the quadratic term uniquely maximal atFor each $j$, maximize the concave function $r_j\log {\pi }_j+q_j\log (1-{\pi }_j)$. Its interior critical point isThe same formula gives the appropriate boundary value when $r_j=0$ or $q_j=0$.