 Solution

Put $G=I-H$. Since $GX=0$, the regression residual vector is $\hat{\epsilon }=G\epsilon$. A linear transformation of a multivariate normal distribution is again multivariate normal, soThis is a singular multivariate normal distribution supported on $\ker X^T$, with dimension $n-p$. In particular,Consequently regression residuals are neither independent nor identically distributed even under the stated model. Their linear constraints include $X^T\hat{\epsilon }=0$, and, if the design matrix contains an intercept, their sum is zero.

Choose an orthonormal basis of the residual space. The coordinates of $G\epsilon$ in that basis are $n-p$ independent $N(0,{\sigma }^2)$ variables, provingFor $n>p$, $s^2$ is an unbiased estimator of ${\sigma }^2$. Orthogonality of $H$ and $G$ gives zero cross-covariance, so the Gaussian fitted and residual vectors are independent. This explains why an appropriate residual-versus-fitted diagnostic has no systematic mean relationship under the model. If $n=p$, no residual information remains for these diagnostics or the variance estimate.

For $h_{ii}<1$, use standardized regression residuals $r_i={\hat{\epsilon }}_i/[s\sqrt{1-h_{ii}}]$ to account for the different residual variances. The diagonal $h_{ii}$ is the regression leverage; $h_{ii}=1$ makes that residual identically zero and cannot be diagnosed by this standardization. With known $\sigma$, the corresponding individually standardized residual has an exact standard normal distribution. With estimated $s$, the internally studentized residual is not exactly normal or a Student t-distribution, and the residuals remain correlated; a normal reference plot is a diagnostic approximation.

Plot the regression residuals or standardized regression residuals against fitted values and each covariate. A persistent curve suggests a misspecified mean, while a fan-shaped spread suggests heteroscedasticity; a scale-location plot helps separate these. A quantile-quantile plot against normal quantiles checks tails, skewness and possible regression outliers. Plot residuals against observation order or time to look for unexplained serial patterns, recognizing that projection already induces the displayed residual covariances. Inspect regression leverage together with Cook's distance for influential observations. Such patterns motivate revising the model; absence of a visible pattern does not prove all its assumptions.