= Representer theorem
{wiki}
Let $\mathcal H$ be a <Reproducing kernel Hilbert space> with <positive-semidefinite kernel> $k$. Suppose an objective has the form
$$
L(f(x_1),\ldots,f(x_n))+\Omega(\|f\|_{\mathcal H}),
$$
where $\Omega$ is strictly increasing. If a minimizer exists, every minimizer lies in the <linear span> of the sections $k(x_i,\cdot)$:
$$
f(\cdot)=\sum_{i=1}^n\alpha_i k(x_i,\cdot).
$$
To prove this, decompose $f=f_\parallel+f_\perp$ into its <orthogonal projection> onto that finite-dimensional <linear span> and its orthogonal remainder. The <reproducing property> gives $f_\perp(x_i)=0$, so removing the remainder preserves the data-dependent term. The <Pythagorean theorem> gives $\|f\|_{\mathcal H}^2=\|f_\parallel\|_{\mathcal H}^2+\|f_\perp\|_{\mathcal H}^2$, so strict increase of $\Omega$ excludes a nonzero remainder at a minimizer.
For merely nondecreasing $\Omega$, projecting an existing minimizer still gives a minimizer in the <linear span>, but other minimizers may remain outside it. The theorem is a finite-representation result; existence of a minimizer needs separate hypotheses.
Back to article page
