= Solution
Put $G=I-H$. Since $GX=0$, the <regression residual> vector is $\widehat\varepsilon=G\varepsilon$. A linear transformation of a <multivariate normal distribution> is again <multivariate normal>, so
$$
\boxed{\widehat\varepsilon\sim N_n(0,\sigma^2G),\qquad G=I-H.}
$$
This is a singular <multivariate normal distribution> supported on $\ker X^T$, with dimension $n-p$. In particular,
$$
\operatorname{Var}(\widehat\varepsilon_i)=\sigma^2(1-h_{ii}),\qquad
\operatorname{Cov}(\widehat\varepsilon_i,\widehat\varepsilon_j)=-\sigma^2h_{ij}\quad(i\ne j).
$$
Consequently <regression residuals> are neither independent nor identically distributed even under the stated model. Their linear constraints include $X^T\widehat\varepsilon=0$, and, if the <design matrix> contains an intercept, their sum is zero.
Choose an <orthonormal basis> of the residual space. The coordinates of $G\varepsilon$ in that basis are $n-p$ <independent> $N(0,\sigma^2)$ variables, proving
$$
\frac{Q(\widehat\beta)}{\sigma^2}\sim\chi^2_{n-p},\qquad
s^2=\frac{Q(\widehat\beta)}{n-p}.
$$
For $n>p$, $s^2$ is an <unbiased estimator> of $\sigma^2$. Orthogonality of $H$ and $G$ gives zero cross-<covariance>, so the <Gaussian> fitted and residual vectors are <independent>. This explains why an appropriate residual-versus-fitted diagnostic has no systematic mean relationship under the model. If $n=p$, no residual information remains for these diagnostics or the variance estimate.
For $h_{ii}<1$, use <standardized regression residuals> $r_i=\widehat\varepsilon_i/[s\sqrt{1-h_{ii}}]$ to account for the different residual <variances>. The diagonal $h_{ii}$ is the <regression leverage>; $h_{ii}=1$ makes that residual identically zero and cannot be diagnosed by this standardization. With known $\sigma$, the corresponding individually standardized residual has an exact <standard normal distribution>. With estimated $s$, the <internally studentized residual> is not exactly normal or a <Student t-distribution>, and the residuals remain correlated; a normal reference plot is a diagnostic approximation.
Plot the <regression residuals> or <standardized regression residuals> against <fitted values> and each covariate. A persistent curve suggests a misspecified mean, while a fan-shaped spread suggests <heteroscedasticity>; a <scale-location plot> helps separate these. A <quantile-quantile plot> against normal quantiles checks tails, skewness and possible <regression outliers>. Plot residuals against observation order or time to look for unexplained serial patterns, recognizing that projection already induces the displayed residual <covariances>. Inspect <regression leverage> together with <Cook's distance> for <influential observations>. Such patterns motivate revising the model; absence of a visible pattern does not prove all its assumptions.
Back to article page
