Past exam of the mathematics course of the University of Cambridge / 2001 / iii / Paper 30 / 1

Let each observation have dimension $p$, and define the sample mean and the divisor-$n$ sample covariance matrix byThe multivariate normal distribution gives the log-likelihoodWriting $y_i-\mu =(y_i-\bar{y})+(\bar{y}-\mu )$ makes the cross terms vanish, and the last sum becomes $n\mathrm{tr}(V^{-1}S)+n(\bar{y}-\mu {)}^T{V}^{-1}(\bar{y}-\mu )$. Since $V$ is a positive-definite matrix, its minimum over the mean is at $\hat{\mu }=\bar{y}$. ConsequentlyWhen $S$ is a positive-definite matrix, $\log \det (V^{-1}S)=\log \det S-\log \det V$. Thuswhere $C^{\prime }$ depends on the data but not on $V$.

For the maximum-likelihood estimator, consider the positive-definite matrix $Q=S^{1/2}{V}^{-1}{S}^{1/2}$, with positive eigenvalues $a_1,\dots ,a_p$. The part of the profile likelihood depending on $V$ is $\frac{n}{2}{\sum }_j(\log a_j-a_j)$. The elementary inequality $\log a-a\le -1$, with equality exactly at $a=1$, shows that the unique maximizing matrix is $Q=I$. HenceThe divisor is $n$, not the unbiased covariance divisor $n-1$. The regularity qualification matters: if $S$ is singular, the displayed logarithm of its determinant is not finite, and no positive-definite covariance maximizes the likelihood. Sending the fitted variance in a null direction of $S$ to zero makes the likelihood unbounded. For a nonsingular normal population, $S$ is positive definite almost surely when $n>p$, which is the setting of the large-sample test.

Under the diagonal covariance restriction, the profile log-likelihood separates into $p$ terms,Differentiating each term gives the restricted maximum-likelihood estimator ${\hat{V}}_0=D=\mathrm{diag}(S_{11},\dots ,S_{pp})$. Both fitted trace terms equal $p$, so the likelihood-ratio test statistic iswhere $R=D^{-1/2}S{D}^{-1/2}$ is the sample correlation matrix. The statistic is nonnegative by the Hadamard inequality; large values indicate dependence. Equivalently, the likelihood ratio is $(\det S/\det D{)}^{n/2}$ and rejection is for small values of this ratio.

For fixed $p$ and a positive-definite true covariance, the Wilks theorem says that twice the maximized log-likelihood difference for nested regular models converges under the null to a chi-squared distribution with degrees of freedom equal to the dimension difference. The unrestricted covariance has $p(p+1)/2$ parameters and the restricted covariance has $p$; the unknown mean has the same $p$ parameters in both. Therefore the Gaussian covariance diagonality likelihood-ratio test of asymptotic level $\alpha$ isWithin the multivariate normal distribution, diagonal covariance also means independent coordinates.