= Solution
Use the <normal linear model> $Y=X\beta+\varepsilon$, with fixed full-column-rank <design matrix> $X$, independent errors $\varepsilon_i\sim N(0,\sigma^2)$, and $\sigma^2>0$. Its <log-likelihood> is
$$
\ell(\beta,\sigma^2)=-\frac n2\log(2\pi\sigma^2)-\frac{(Y-X\beta)^T(Y-X\beta)}{2\sigma^2}.
$$
For fixed $\sigma^2$, differentiating with respect to $\beta$ gives $X^T(Y-X\widehat\beta)=0$. Since $X^TX$ is positive-definite, the unique least-squares estimate is
$$
\boxed{\widehat\beta=(X^TX)^{-1}X^TY.}
$$
Let $\operatorname{RSS}=\|Y-X\widehat\beta\|^2$. The <variance> score equation is $-n/(2\sigma^2)+\operatorname{RSS}/(2\sigma^4)=0$, giving
$$
\boxed{\widehat\sigma^2_{\rm ML}=\operatorname{RSS}/n.}
$$
This is the <maximum-likelihood estimator>, whereas the <residual standard error> reported by linear-regression software is $s=\sqrt{\operatorname{RSS}/(n-p)}$. The latter uses the unbiased <residual estimate of Gaussian noise variance>. These formulas assume $n>p$ and positive RSS; exact interpolation gives a <variance>-boundary degeneracy rather than an interior maximum.
Back to article page
