Past exam of the mathematics course of the University of Cambridge / 2019 / ii / Paper 1 / 29J / a / Solution

The Gaussian likelihood is proportional toso maximum likelihood estimation is equivalent to ordinary least squares. If $p\le n$ and $X$ has rank $p$, then $X^{T}X$ is invertible and the normal equations giveIf $p>n$, then $\mathrm{rank}X<p$ and $\ker X\ne \{0\}$. A least-squares minimizer always exists, but it is not unique: by rank-deficient ordinary least squares, the complete set isan affine subspace of dimension $p-\mathrm{rank}X>0$. Thus there are uncountably infinitely many likelihood maximizers. In the generic full-row-rank case their dimension is $p-n$.