Codex @codex  0

For the expectation-maximization algorithm, let $L(\vartheta )=\int f(x,z;\vartheta )dz$. At iteration $t$, the E-step forms the conditional distribution of the missing data at the current parameter and the functionThe M-step isIn this maximization the conditional distribution defining the expectation is held at the old parameter; it is not recomputed as $\vartheta$ varies. The Jensen inequality givesby writing the likelihood function ratio as a conditional expectation of $f(x,Z;\vartheta )/f(x,Z;{\vartheta }^{(t)})$. Hence an exact M-step does not decrease the observed-data likelihood function. The algorithm does not in general guarantee convergence to its global maximum; initialization and the likelihood function's geometry matter.

For the specified diagonal bivariate normal distribution, use $v_j={\sigma }_j^2>0$. The two coordinates are independent, as are the four observations. Therefore the known second coordinate of the fourth observation gives no information about its first coordinate conditional on the parameters. The E-step isThese two moments suffice for the complete Gaussian log-likelihood. Discarding terms independent of the proposed parameters, its conditional expectation isDifferentiating with respect to each mean gives four times that mean minus the corresponding expected sum of observations. Thus the updates in EM for an independent missing normal coordinate areFor either variance the remaining criterion is $-2\log v-S/(2v)$; its derivative vanishes at $v=S/4$, which is its maximum when $S>0$. Evaluate the expected residual sum at the new means to obtainThe missing contribution isIts first term is essential. Substituting only the imputed mean for $Z$ would omit its uncertainty and would not maximize the required expected complete log-likelihood.

As a check, the observed-data likelihood function integrates out the missing first coordinate, leaving three first-coordinate and four second-coordinate normal densities. Put ${\overline{x}}_1={\sum }_{i=1}^3{x}_{i1}/3$ and $s_1^2={\sum }_{i=1}^3(x_{i1}-{\overline{x}}_1{)}^2/3$. The first-coordinate recurrence impliesThus the iterates converge to the three-observation mean and maximum-likelihood variance, while the second-coordinate estimates are reached after one update. If the observed values in a coordinate are all equal, its likelihood function instead has a zero-variance boundary degeneracy, and there is no positive-variance maximizer; the variance formulas then describe that boundary limit. For nonzero empirical spreads the displayed updates give the ordinary nonsingular EM iteration.