Past exam of the mathematics course of the University of Cambridge / 2003 / iii / Paper 42 / 1

Use the divisor-$n$ sample covariance matrix, since this is the normalization that enters the multivariate normal likelihood function:For a covariance matrix that is a positive-definite matrix $V$, the product of the multivariate normal densities givesTo separate the sample mean from the sample covariance matrix, expand $x_j-\mu =(x_j-\bar{x})+(\bar{x}-\mu )$. The cross terms sum to zero because ${\sum }_j(x_j-\bar{x})=0$. Also $u^T{V}^{-1}u=\mathrm{tr}(V^{-1}u{u}^T)$ by the trace identity. ThereforeThe printed expression suppresses the parameter-independent constant $np\log (2\pi )$; dropping this constant has no effect on maximum likelihood estimation or the likelihood-ratio test.

With the divisor-$n$ sample covariance matrix defined above, the maximum-likelihood estimators areTheir joint sampling distribution is specified byHere $W_p(\nu ,V)$ uses the Wishart distribution convention $\mathbb{E}W=\nu V$. Thus the covariance maximum-likelihood estimator has expected value $(n-1)V/n$; the unbiased estimator of the covariance matrix is $nS/(n-1)$. These distributions follow from orthogonal projections of the jointly multivariate normal observations onto the constant-observation direction and its orthogonal complement, though no proof is required here. The covariance maximum in the positive-definite parameter space requires $S$ nonsingular, which holds almost surely when $n>p$ and $V$ is positive definite. Otherwise the rank of a centered sample covariance matrix shows why the usual unrestricted finite maximum does not exist.

Under the unrestricted model, insert the maximum-likelihood estimates $(\bar{x},S)$. Under the diagonal-covariance restriction, the quadratic form is minimized over the mean at $\bar{x}$, and the remaining objective separates over the diagonal entries:Differentiating in $v_j>0$ gives ${\hat{v}}_j=S_{jj}$; the derivative changes from negative to positive there. Put $D=\mathrm{diag}(S_{11},\dots ,S_{pp})$. The fitted trace term is $p$ under either model, since $\mathrm{tr}(D^{-1}S)=p=\mathrm{tr}(S^{-1}S)$. Consequently the likelihood-ratio test statistic is determined byThis is the Gaussian diagonal-covariance likelihood-ratio test. Reject for sufficiently small $\log |R|$, equivalently for large $-n\log |R|$. Under the regular large-sample calibration, Wilks theorem gives $-n\log |R|\Rightarrow {\chi }_{p(p-1)/2}^2$, since diagonality removes that many free covariance parameters. The exact critical constant can instead be obtained from the null distribution at the actual sample size.

For $p=2$, write the sample correlation coefficient as $r$. Then $|R|=1-r^2$, so the likelihood-ratio test rejects for large $|r|$, not merely for large positive $r$. Under the null, the two variables have a bivariate normal distribution and are independent, and the exact test isIndeed, condition on the centered first-variable observation vector. Decompose the second centered Gaussian vector into its component along that direction and the $n-2$-dimensional orthogonal complement. Their squared lengths give independent ${\chi }_1^2$ and ${\chi }_{n-2}^2$ variables. Their ratio shows that $r\sqrt{(n-2)/(1-r^2)}$ has Student's t-distribution with $n-2$ statistical degrees of freedom.