Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 41 / 5

Let $X$ denote the observed data, $Z$ the unobserved data and $\theta$ the model parameters. The expectation-maximization algorithm works with the complete-data log-likelihood ${\ell }_c(\theta ;X,Z)$. Given ${\theta }^{(r)}$, its E-step computeswhere the conditional expectation uses the old parameter, while $\theta$ remains the argument to maximize. Its M-step chooses ${\theta }^{(r+1)}$ maximizing this function over the parameter space. EM likelihood monotonicity ensures that an M-step increasing $Q$ cannot decrease the observed-data log-likelihood. It does not guarantee a global maximum, so convergence and any parameter constraints must still be checked.

The sum of the two independent exponential times has a hypoexponential distribution. By convolution of probability densities, for $t>0$ and $d={\lambda }_1-{\lambda }_2>0$,Thereforeand $g(t)=0$ for $t<0$. This is a nonnegative normalized probability density function. At the equal-rate boundary it tends to ${\lambda }^{2}t{e}^{-\lambda t}$, a shape-two Gamma distribution.

Put $A={\sum }_{i=1}^3{S}_{i1}$, $B={\sum }_{i=1}^3{S}_{i2}$ and $d={\lambda }_1-{\lambda }_2$. Independence of the pairs makes their likelihood contributions multiply. The three fully observed pairs contribute $({\lambda }_1{\lambda }_2{)}^3{e}^{-{\lambda }_{1}A-{\lambda }_{2}B}$; each remaining pair contributes its hypoexponential distribution density. Thus the observed-data log-likelihood isAn equivalent expression, convenient for calculations, isThe apparent singularity as $d\downarrow 0$ cancels between the last sum and $-5\log d$. The resulting equal-rate limit is finite for positive observed times.

Take the missing strong-swimmer times $U_j=S_{j1}$, $j=4,\dots ,8$, as the latent variables. Their partners' times are $T_j-U_j$. The complete-data log-likelihood isFor the E-step of the expectation-maximization algorithm, use the conditional split of two exponential times. At the old rates, with $d_r={\lambda }_1^{(r)}-{\lambda }_2^{(r)}>0$,This is a truncated exponential distribution, whose conditional expectation isThe displayed mean follows by integrating $u{e}^{-d_{r}u}$ against the conditional density; the missing weak-swimmer mean is $T_j-m_j^{(r)}$. As $d_r\to 0$, the conditional density becomes uniform on $(0,T_j)$ and $m_j^{(r)}\to T_j/2$.

Write $U_r=A+{\sum }_j{m}_j^{(r)}$ and $V_r=B+{\sum }_j(T_j-m_j^{(r)})$. The E-step objective isSetting its partial derivatives to zero gives the unconstrained M-stepThese are eight divided by the completed expected totals for the corresponding swimmers; imputing only observed pair totals without the conditional split would not perform the E-step.

The stated ordering must also be enforced. The ordered-rate exponential M-step accepts these updates when $U_r<V_r$, so ${\lambda }_1^{(r+1)}>{\lambda }_2^{(r+1)}$. Otherwise its maximum on the closed constraint ${\lambda }_1\ge {\lambda }_2$ lies atUnder a strictly open constraint ${\lambda }_1>{\lambda }_2$, a boundary optimum is only a supremum; a finite interior maximizer is not assured for every possible sample. One should report that situation rather than swap the labeled swimmer groups. Starting from positive ordered rates, iterate the E-step and constrained M-step until the parameter changes and observed log-likelihood improvement are small. EM likelihood monotonicity applies to every exact constrained M-step. No numerical swimming times are supplied, so the data-dependent updates, rather than fabricated numerical estimates, are the answer.