Because the columns of the <design matrix> are centred, <ridge regression> with an unpenalized intercept solves
$$
(\widehat\alpha_\lambda,\widehat\beta_\lambda)
=\underset{\alpha\in\mathbb R,\,\beta\in\mathbb R^p}{\operatorname{argmin}}
\left\{\|Y-\alpha\mathbf1-X\beta\|_2^2
+\lambda\|\beta\|_2^2\right\}.
$$
The <normal equations> give, for $\lambda>0$,
$$
\widehat\alpha_\lambda=\overline Y,
\qquad
\widehat\beta_\lambda=(X^TX+\lambda I_p)^{-1}X^T(Y-\overline Y\mathbf1)
=(X^TX+\lambda I_p)^{-1}X^TY.
$$
The <fitted values> are consequently
$$
\widehat Y_\lambda
=\overline Y\mathbf1+X(X^TX+\lambda I_p)^{-1}X^TY.
$$
If the objective is normalized by $n$, the same formulas hold after replacing $\lambda$ by $n\lambda$.

Solution

Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 218 / 2 / a / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2024/iii/paper-218/2/a/solution