Past exam of the mathematics course of the University of Oxford / Undated / Prelims / Mathematics III resit / 7 / c

For the random vector $\mathbf{ϵ}=({ϵ}_1,\dots ,{ϵ}_n{)}^{\top }$, its covariance matrix isIts diagonal entries are variances, and its off-diagonal entries measure pairwise covariance. Finite second moments make every entry well defined.

Independence of the centered standard normal random variables gives $\mathbb{E}{ϵ}_i=0$ for every $i$. AlsoFor nonadjacent indices, the underlying sets of $\eta$ variables are disjoint, so the errors are independent and have zero covariance. For adjacent indices with $i\ge 2$,The first adjacent pair separately gives $\mathbb{E}[{ϵ}_1{ϵ}_2]=\mathbb{E}[{\eta }_1^2{\eta }_2]=0$. HenceThese adjacent products of independent Gaussian variables are uncorrelated random variables even though adjacent errors share a factor and are dependent.

The least-squares estimators depend only on the observed responses and design, so the formulas in i still hold. Their variances and covariance need only the error covariance matrix, now $I_n$. ThereforeFor example the first formula follows from $\mathrm{Var}({\sum }_i{x}_i{ϵ}_i)={\sum }_i{x}_i^2$, and the second from ${\sum }_i{x}_i=0$. The formulas remain the same as before with ${\sigma }^2=1$; this calculation does not establish independence or a normal distribution for the estimators.

Compute fitted responses ${\hat{y}}_i=\hat{\alpha }+\hat{\beta }{x}_i$ and regression residuals $e_i=y_i-{\hat{y}}_i$. A conventional residual plot shows $e_i$ against ${\hat{y}}_i$ or $x_i$; plotting against observation order additionally looks for runs. One checks for curvature, changing spread and other structure rather than random scatter around zero.

An ordinary residual plot need not reveal this failure of independence. The errors have constant mean zero, constant variance one and zero pairwise covariances, so the usual trend and changing-variance clues need not appear. A visually patternless residual plot cannot validate independence; fitted regression residuals themselves also share the fitted coefficients even when the true errors are independent.

There is detectable dependence in the magnitudes. The Gaussian fourth moment is $\mathbb{E}{\eta }_i^4=3$, giving for every adjacent pairFor $i=1$, the product is ${\eta }_1^4{\eta }_2^2$; for $i\ge 2$, it is ${\eta }_{i-1}^2{\eta }_i^4{\eta }_{i+1}^2$, with the same expected value 3. Large magnitudes tend to occur together. A lagged plot of squared regression residuals or a test directed at dependence in their squares is more informative. Thus recognition is possible with suitable diagnostics, but is not guaranteed by the standard residual plot requested here.