Maximum-likelihood covariance estimator for centered Gaussian data (source code)

= Maximum-likelihood covariance estimator for centered Gaussian data
{title2=$\widehat\Sigma$}

For <independent random variables>[independent] Gaussian <random vectors> $Z_i\sim N_p(0,\Sigma)$ with known zero <mean> and <positive-definite matrix>[positive-definite] <covariance matrix> $\Sigma$, the candidate maximum-likelihood <covariance> <estimator> is
$$
\widehat\Sigma=S=\frac1n\sum_{i=1}^nZ_iZ_i^{\mathsf T}.
$$
When $S$ is <positive-definite matrix>[positive definite], the Gaussian <log-likelihood function>[log likelihood], up to a constant, is $-\tfrac n2[\log\det\Sigma+\operatorname{tr}(\Sigma^{-1}S)]$. Writing the precision <matrix> as $\Omega=\Sigma^{-1}$ gives the concave objective $\tfrac n2[\log\det\Omega-\operatorname{tr}(\Omega S)]$, whose <derivative> vanishes uniquely at $\Omega=S^{-1}$. This proves the displayed <estimator>. It is unbiased because the <mean> is known: $\mathbb ES=\Sigma$.

If the <statistical sample>[sample] <vector>[vectors] do not span the whole $p$-dimensional space, $S$ is singular. Shrinking a candidate <covariance> eigenvalue to zero along an unobserved direction makes the <likelihood function>[likelihood] unbounded, so there is no maximum within the <positive-definite matrix>[positive-definite] <parameter space>. In particular, $n<p$ cannot yield a <positive-definite matrix>[positive-definite] maximum-likelihood estimate. For a <positive-definite matrix>[positive-definite] Gaussian population and $n\geq p$, the <vector>[vectors] span the space <almost surely>. When the <mean> is instead estimated from the same data, centering changes the rank condition and the unbiased <covariance> uses denominator $n-1$ rather than $n$.