Past exam of the mathematics course of the University of Cambridge / 2001 / iii / Paper 30 / 1 / i

Let each observation have dimension $p$, and define the sample mean and the divisor-$n$ sample covariance matrix byThe multivariate normal distribution gives the log-likelihoodWriting $y_i-\mu =(y_i-\bar{y})+(\bar{y}-\mu )$ makes the cross terms vanish, and the last sum becomes $n\mathrm{tr}(V^{-1}S)+n(\bar{y}-\mu {)}^T{V}^{-1}(\bar{y}-\mu )$. Since $V$ is a positive-definite matrix, its minimum over the mean is at $\hat{\mu }=\bar{y}$. ConsequentlyWhen $S$ is a positive-definite matrix, $\log \det (V^{-1}S)=\log \det S-\log \det V$. Thuswhere $C^{\prime }$ depends on the data but not on $V$.

For the maximum-likelihood estimator, consider the positive-definite matrix $Q=S^{1/2}{V}^{-1}{S}^{1/2}$, with positive eigenvalues $a_1,\dots ,a_p$. The part of the profile likelihood depending on $V$ is $\frac{n}{2}{\sum }_j(\log a_j-a_j)$. The elementary inequality $\log a-a\le -1$, with equality exactly at $a=1$, shows that the unique maximizing matrix is $Q=I$. HenceThe divisor is $n$, not the unbiased covariance divisor $n-1$. The regularity qualification matters: if $S$ is singular, the displayed logarithm of its determinant is not finite, and no positive-definite covariance maximizes the likelihood. Sending the fitted variance in a null direction of $S$ to zero makes the likelihood unbounded. For a nonsingular normal population, $S$ is positive definite almost surely when $n>p$, which is the setting of the large-sample test.