= Solution
Put $G=X^TX$ and $H=XG^{-1}X^T$. The <rank of a matrix> assumption makes $G$ invertible, and $H$ is the <orthogonal projection matrix> onto the column space of the <design matrix>. The <Gaussian likelihood> has <log-likelihood>
$$
\ell(b,v)=-\frac n2\log(2\pi v)-\frac{\|Y-Xb\|^2}{2v},\qquad v>0.
$$
Minimizing the squared <Euclidean norm> gives the <ordinary least squares> normal equations $G\widehat\beta=X^TY$. Maximizing over $v$ then gives the <maximum-likelihood estimators>
$$
\boxed{\widehat\beta=G^{-1}X^TY,\qquad \widehat\sigma^2=\frac{\mathrm{RSS}}n,\qquad \mathrm{RSS}=Y^T(I-H)Y.}
$$
The divisor for the <maximum-likelihood estimator> of the <variance> is $n$, whereas the unbiased residual <variance> estimate is $s^2=\mathrm{RSS}/(n-p)$. The exceptional event $\mathrm{RSS}=0$ has probability zero under the <Gaussian linear model> with $\sigma^2>0$; on that event the <likelihood> is unbounded as $v\downarrow0$.
For the joint <sampling distribution>, write $\widehat\beta-\beta=G^{-1}X^T\epsilon$ and $Y-X\widehat\beta=(I-H)\epsilon$. Their cross-<covariance matrix> vanishes because $X^T(I-H)=0$. These two vectors are jointly <multivariate normal>, so they are <independent random variables>. The residual <orthogonal projection> has rank $n-p$, hence the standard <chi-squared distribution> result gives
$$
\boxed{\widehat\beta\sim N_p(\beta,\sigma^2G^{-1}),\qquad \frac{n\widehat\sigma^2}{\sigma^2}\sim\chi^2_{n-p},\qquad \widehat\beta\ \text{and}\ \widehat\sigma^2\ \text{independent}.}
$$
Thus these two displayed marginal laws, with their <independence>, specify the full joint distribution.
The <quadratic form>
$$
\frac{(\widehat\beta-\beta)^TG(\widehat\beta-\beta)}{\sigma^2}
$$
has a <chi-squared distribution> with $p$ degrees of freedom and is independent of $\mathrm{RSS}/\sigma^2$. Their ratio therefore supplies the exact <F-distribution> pivot
$$
T(\beta)=\frac{(\widehat\beta-\beta)^TG(\widehat\beta-\beta)/p}{\mathrm{RSS}/(n-p)}\sim F_{p,n-p}.
$$
If $f_{1-\alpha}$ is the $(1-\alpha)$ quantile of that <F-distribution>, inversion gives the <normal linear-model confidence ellipsoid>
$$
\boxed{C_{1-\alpha}(Y)=\{b\in\mathbb R^p:(\widehat\beta-b)^TG(\widehat\beta-b)\le p s^2 f_{1-\alpha}\}.}
$$
For every $\beta$ and every $\sigma^2>0$, the probability that this <confidence set> contains $\beta$ is exactly $1-\alpha$. To test the <null hypothesis>, reject precisely when $\beta_0$ lies outside this <confidence ellipsoid>, equivalently when $T(\beta_0)>f_{1-\alpha}$. Under the <null hypothesis> the rejection probability is exactly $\alpha$, independent of the unknown <variance>.
For the numerical calculation, the second column of the <design matrix> sums to zero and its sum of squares is $60$. Direct calculation from the observations gives
$$
G=\begin{pmatrix}9&0\\0&60\end{pmatrix},\qquad X^TY=\begin{pmatrix}0\\30\end{pmatrix},\qquad Y^TY=22.
$$
Consequently the <ordinary least squares> estimate, <residual sum of squares> and unbiased residual <variance> are
$$
\widehat\beta=\begin{pmatrix}0\\1/2\end{pmatrix},\qquad \mathrm{RSS}=22-(X^TY)^TG^{-1}(X^TY)=22-15=7,\qquad s^2=7/7=1.
$$
The <maximum-likelihood estimator> of the <variance> is instead $7/9$. At the proposed null value the <F-distribution> statistic is
$$
\boxed{T(0)=\frac{15/2}{7/7}=7.5>4.74.}
$$
Therefore \b[reject the null hypothesis at the 5% level]. The relevant critical value has $2$ numerator and $7$ residual degrees of freedom.
Back to article page
