The <kernel ridge regression> estimator is
$$
\widehat f_\lambda
=\underset{f\in\mathcal H}{\operatorname{argmin}}
\left\{\frac1n\sum_{i=1}^n(Y_i-f(x_i))^2+\lambda\lVert f\rVert_{\mathcal H}^2\right\}.
$$
By the <representer theorem>, $\widehat f_\lambda=\sum_{j=1}^n\alpha_jk(x_j,\cdot)$. If $K_{ij}=k(x_i,x_j)$, substitution and differentiation give
$$
(K+n\lambda I)\alpha=Y.
$$
Thus
$$
\alpha=(K+n\lambda I)^{-1}Y,
\qquad
\widehat Y=H_\lambda Y,
\qquad
H_\lambda=K(K+n\lambda I)^{-1}.
$$
The matrix $H_\lambda$ is the <kernel-ridge hat matrix>.

Solution

Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 205 / 5 / a / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2024/iii/paper-205/5/a/solution