= Solution
The <Gaussian linear model> has log <likelihood function>, up to an additive constant,
$$
\ell(\theta,\sigma^2)=-\frac n2\log\sigma^2-
\frac1{2\sigma^2}(Y-X\theta)^T(Y-X\theta).
$$
For each fixed positive $\sigma^2$, maximize this by minimizing the residual sum of squares. Differentiation gives the <normal equations for linear least squares>[normal equations]
$$
X^TX\widehat\theta=X^TY.
$$
The full column rank of $X$ makes $X^TX$ <positive-definite matrix>[positive definite], hence invertible, so
$$
\boxed{\widehat\theta=(X^TX)^{-1}X^TY.}
$$
The quadratic objective is strictly convex, which proves uniqueness. The joint <maximum likelihood estimation>[maximum-likelihood estimate] has the same coefficient estimate, except on the <probability>-zero event of an exactly zero residual when the positive-<variance> likelihood has no finite joint maximizer.
Back to article page
