 Solution

ID: past-exam-of-the-options-ii-course-of-the-university-of-oxford/2009/o/o4/solution

Minimizing the sum of squared residuals gives the normal equations for linear least squares,

X^{T} X θ = X^{T} Y

X^{T} X

positive definite and invertible, so the unique minimizer is

θ = A Y, A = (X^{T} X)^{- 1} X^{T} .

Because

E Y = Xθ

and

Cov (Y) = σ^{2} I_{n}

, one has

A X = I_{p}

and

E θ = θ, Cov (θ) = σ^{2} A A^{T} = σ^{2} (X^{T} X)^{- 1} .

For the alternative linear estimator the corresponding general formulas are

E (D Y) = D Xθ, Cov (D Y) = σ^{2} D D^{T} .

No normality assumption on the errors is needed for these claims.

 New to topics? Read the docs here!