= Solution
The Gaussian log <likelihood function>, apart from a constant, is
$$
\ell(\beta,\sigma^2)=-\frac n2\log\sigma^2-\frac{\|Y-X\beta\|^2}{2\sigma^2}.
$$
Differentiating in $\beta$ gives $X^{\mathsf T}(Y-X\widehat\beta)=0$. Full column rank makes $X^{\mathsf T}X$ invertible, so
$$
\boxed{\widehat\beta=(X^{\mathsf T}X)^{-1}X^{\mathsf T}Y,\qquad
\widehat\beta\sim N_p(\beta,\sigma^2(X^{\mathsf T}X)^{-1})}.
$$
The distribution follows by applying this linear map to the Gaussian error vector. Differentiating the profiled <likelihood function> in $\sigma^2$ gives
$$
\boxed{\widehat\sigma^2=\frac1n\|Y-X\widehat\beta\|^2}.
$$
For its distribution, $P=X(X^{\mathsf T}X)^{-1}X^{\mathsf T}$ is symmetric idempotent, and the residual is $(I-P)\epsilon$. The needed form of <Cochran's theorem> says that quadratic forms in a standard Gaussian vector associated with orthogonal symmetric projection matrices summing to identity are independent random variables with <chi-squared distributions> with degrees equal to their ranks. Applying it to $P$ and $I-P$ gives
$$
\boxed{\frac{n\widehat\sigma^2}{\sigma^2}\sim\chi^2_{n-p}}.
$$
Thus the maximum-likelihood variance estimate uses divisor $n$ and has expectation $(n-p)\sigma^2/n$; the unbiased residual estimate would use divisor $n-p$. The fitted coefficient and residual quadratic form are also independent.
Back to article page
