= Kernel ridge regression
<Kernel ridge regression> fits a <function> $f$ in a <Reproducing kernel Hilbert space> by minimizing
$$
\frac1n\sum_{i=1}^n(Y_i-f(x_i))^2+\lambda\|f\|_{\mathcal H}^2,\qquad\lambda>0.
$$
The first term rewards fit to the observations; the second penalizes the <norm>. The <representer theorem> gives $f(x)=\sum_i\alpha_i k(x_i,x)$. With <kernel matrix> $K$, this becomes
$$
\frac1n\|Y-K\alpha\|^2+\lambda\alpha^TK\alpha.
$$
One coefficient solution is $\alpha=(K+n\lambda I)^{-1}Y$, and the fitted observation vector is $K\alpha$. The positive ridge makes this displayed inverse exist even when $K$ is singular; different coefficient vectors in the <null space> of $K$ represent the same fitted <function>.
For a single observation with $k(x_1,x_1)=1$ and $Y_1=2$, the coefficient is $2/(1+\lambda)$ and the prediction is $f(x)=2k(x_1,x)/(1+\lambda)$. This exposes both shrinkage and how the <positive-semidefinite kernel> transfers information to a new point. The $n\lambda$ factor depends on the chosen normalization of the squared-error term.
Back to article page
