Past exam of the mathematics course of the University of Cambridge / 2004 / iii / Paper 40 / 6

The parameter range is $0\le \theta \le 1$. Omitting data-only constants from the multinomial distribution likelihood, its log isFor an interior maximizer, the likelihood score must vanish:Multiply by $\theta (1-\theta )(7+4\theta )$ and collect powers to obtainThe likelihood is concave: its second derivative in the interior isTherefore an admissible interior root is the unique maximum. Endpoints must also be handled if some counts vanish. If $x_3>0$ and $x_1+x_2>0$, both endpoints have zero likelihood and the root is interior. If the maximum is at zero then $x_3=0$, and zero also satisfies the displayed polynomial; a maximum at one requires $x_1+x_2=0$, and one likewise satisfies it. Multiplying the score can introduce other endpoint roots, so not every polynomial root is an MLE: evaluate the actual likelihood on the feasible candidates.

Let $Z$ be the unobserved part of the fourth count associated with the component of probability $\theta /3$. Its complementary subcount has probability $7/12$. Conditional on the observed fourth count and a parameter value,The complete-data log-likelihood now has the simple formThe constant term can depend on the completed counts, but not on $\theta$. The split therefore converts the troublesome $\log (7+4\theta )$ term into a binomial-like maximization problem. Since $Z$ is unobserved, it should be averaged conditionally in an EM algorithm, not guessed or treated as observed.

For observed data $x$ and missing data $z$, let $p_{\theta }(x,z)$ be the complete-data model. Starting at ${\theta }^{(0)}$, the expectation-maximization algorithm alternatesThe E-step computes the conditional expectation of the complete-data log-likelihood, using the current parameter for the missing-data conditional distribution. The M-step maximizes that expected log-likelihood over a new parameter. Maximizing the likelihood of imputed mean data is equivalent only when the complete-data log-likelihood has the necessary linear dependence on the missing sufficient statistics.

For the monotonicity argument, let $q_r(z)=p_{{\theta }^{(r)}}(z\mid x)$. Jensen inequality givesTherefore an M-step increasing $Q$ cannot decrease the observed-data likelihood. Standard support and integrability conditions are understood. EM need not find a global maximum for a general model; its behavior depends on the objective and starting point.

For these counts, start with any $0<{\theta }^{(0)}<1$. The conditional missing count from part (ii) has meanThe E-step objective isIts strictly concave M-step is explicit:Equivalently, the update map is $F(\theta )=(35+360\theta )/(245+480\theta )$. Its fixed-point equation isThe unique root in $(0,1)$ is the maximum-likelihood estimateFor example, starting at $1/2$ gives $0.443299$, then approximately $0.425065$, and subsequent iterates approach the displayed root. Convergence is also transparent without relying only on the general EM theorem: $F$ is increasing andThis is positive below the feasible fixed point and negative above it. Monotonicity of $F$ prevents crossing that fixed point, so the iterates converge monotonically toward it from either side. The original log-likelihood is strictly concave and diverges to minus infinity at both endpoints, establishing that the fixed point is the unique global maximum.