Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2020/ib/paper-2/18h/a/solution

Because has full column rank, is invertible. The ordinary least squares estimator is
It satisfies the normal equations
so the residual is orthogonal to the column space of . For any ,
The two terms are orthogonal, and the Pythagorean theorem in an inner-product space gives
Thus minimizes the least-squares objective.

New to topics? Read the docs here!