Maximum-likelihood covariance estimator for centered Gaussian data

ID: maximum-likelihood-covariance-estimator-for-centered-gaussian-data

For independent Gaussian random vectors with known zero mean and positive-definite covariance matrix , the candidate maximum-likelihood covariance estimator is
When is positive definite, the Gaussian log likelihood, up to a constant, is . Writing the precision matrix as gives the concave objective , whose derivative vanishes uniquely at . This proves the displayed estimator. It is unbiased because the mean is known: .
If the sample vectors do not span the whole -dimensional space, is singular. Shrinking a candidate covariance eigenvalue to zero along an unobserved direction makes the likelihood unbounded, so there is no maximum within the positive-definite parameter space. In particular, cannot yield a positive-definite maximum-likelihood estimate. For a positive-definite Gaussian population and , the vectors span the space almost surely. When the mean is instead estimated from the same data, centering changes the rank condition and the unbiased covariance uses denominator rather than .

New to topics? Read the docs here!