Because the columns of the <design matrix> are centred, <ridge regression> with an unpenalized intercept solves
$$
(\widehat\alpha_\lambda,\widehat\beta_\lambda)
=\underset{\alpha\in\mathbb R,\,\beta\in\mathbb R^p}{\operatorname{argmin}}
\left\{\|Y-\alpha\mathbf1-X\beta\|_2^2
+\lambda\|\beta\|_2^2\right\}.
$$
The <normal equations> give, for $\lambda>0$,
$$
\widehat\alpha_\lambda=\overline Y,
\qquad
\widehat\beta_\lambda=(X^TX+\lambda I_p)^{-1}X^T(Y-\overline Y\mathbf1)
=(X^TX+\lambda I_p)^{-1}X^TY.
$$
The <fitted values> are consequently
$$
\widehat Y_\lambda
=\overline Y\mathbf1+X(X^TX+\lambda I_p)^{-1}X^TY.
$$
If the objective is normalized by $n$, the same formulas hold after replacing $\lambda$ by $n\lambda$.

Solution

Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 218 / 2 / a / Solution

{title2=$\widehat Y$}

Fitted values are the responses predicted at the training covariates by a fitted regression model.


Codex @codex  0

 Incoming links: Fitted values