= Solution
For the <expectation-maximization algorithm>, let $L(\vartheta)=\int f(x,z;\vartheta)\,dz$. At iteration $t$, the E-step forms the <conditional distribution> of the missing data at the current parameter and the function
$$
Q(\vartheta\mid\vartheta^{(t)})=
\mathbb E_{\vartheta^{(t)}}[\log f(x,Z;\vartheta)\mid x].
$$
The M-step is
$$
\boxed{\vartheta^{(t+1)}\in\operatorname*{arg\,max}_{\vartheta}
Q(\vartheta\mid\vartheta^{(t)}).}
$$
In this maximization the <conditional distribution> defining the expectation is held at the old parameter; it is not recomputed as $\vartheta$ varies. The <Jensen inequality> gives
$$
\log L(\vartheta)-\log L(\vartheta^{(t)})
\ge Q(\vartheta\mid\vartheta^{(t)})-Q(\vartheta^{(t)}\mid\vartheta^{(t)}),
$$
by writing the <likelihood function> ratio as a <conditional expectation> of $f(x,Z;\vartheta)/f(x,Z;\vartheta^{(t)})$. Hence an exact M-step does not decrease the observed-data <likelihood function>. The algorithm does not in general guarantee convergence to its global maximum; initialization and the <likelihood function>'s geometry matter.
For the specified diagonal <bivariate normal distribution>, use $v_j=\sigma_j^2>0$. The two coordinates are independent, as are the four observations. Therefore the known second coordinate of the fourth observation gives no information about its first coordinate conditional on the parameters. The E-step is
$$
Z\mid x_{\mathrm{obs}},\vartheta^{(t)}\sim N(\mu_1^{(t)},v_1^{(t)}),
$$
$$
a_t:=\mathbb E_tZ=\mu_1^{(t)},\qquad
b_t:=\mathbb E_tZ^2=(\mu_1^{(t)})^2+v_1^{(t)}.
$$
These two moments suffice for the complete <Gaussian> log-likelihood. Discarding terms independent of the proposed parameters, its <conditional expectation> is
$$
\begin{aligned}
Q(\mu_1,\mu_2,v_1,v_2\mid\vartheta^{(t)})
={}&-2\log v_1-\frac1{2v_1}
\left[\sum_{i=1}^3(x_{i1}-\mu_1)^2+b_t-2\mu_1a_t+\mu_1^2\right]\\
&-2\log v_2-\frac1{2v_2}\sum_{i=1}^4(x_{i2}-\mu_2)^2.
\end{aligned}
$$
Differentiating with respect to each mean gives four times that mean minus the corresponding expected sum of observations. Thus the updates in <EM for an independent missing normal coordinate> are
$$
\boxed{\mu_1^{(t+1)}=\frac{x_{11}+x_{21}+x_{31}+\mu_1^{(t)}}4,\qquad
\mu_2^{(t+1)}=\frac{x_{12}+x_{22}+x_{32}+x_{42}}4.}
$$
For either <variance> the remaining criterion is $-2\log v-S/(2v)$; its derivative vanishes at $v=S/4$, which is its maximum when $S>0$. Evaluate the expected residual sum at the new means to obtain
$$
\boxed{v_1^{(t+1)}=\frac14\left[
\sum_{i=1}^3(x_{i1}-\mu_1^{(t+1)})^2
+v_1^{(t)}+(\mu_1^{(t)}-\mu_1^{(t+1)})^2\right],}
$$
$$
\boxed{v_2^{(t+1)}=\frac14\sum_{i=1}^4(x_{i2}-\mu_2^{(t+1)})^2.}
$$
The missing contribution is
$$
\mathbb E_t[(Z-\mu_1^{(t+1)})^2]
=v_1^{(t)}+(\mu_1^{(t)}-\mu_1^{(t+1)})^2.
$$
Its first term is essential. Substituting only the imputed mean for $Z$ would omit its uncertainty and would not maximize the required expected complete log-likelihood.
As a check, the observed-data <likelihood function> integrates out the missing first coordinate, leaving three first-coordinate and four second-coordinate normal densities. Put $\overline x_1=\sum_{i=1}^3x_{i1}/3$ and $s_1^2=\sum_{i=1}^3(x_{i1}-\overline x_1)^2/3$. The first-coordinate recurrence implies
$$
\mu_1^{(t+1)}-\overline x_1=\frac14(\mu_1^{(t)}-\overline x_1),
$$
$$
v_1^{(t+1)}=\frac34s_1^2+\frac14v_1^{(t)}
+\frac3{16}(\mu_1^{(t)}-\overline x_1)^2.
$$
Thus the iterates converge to the three-observation mean and maximum-likelihood <variance>, while the second-coordinate estimates are reached after one update. If the observed values in a coordinate are all equal, its <likelihood function> instead has a zero-variance boundary degeneracy, and there is no positive-variance maximizer; the <variance> formulas then describe that boundary limit. For nonzero empirical spreads the displayed updates give the ordinary nonsingular EM iteration.
Back to article page
