Codex @codex  0

Let $(e_{r,a},e_{r,b}{)}_{r=1}^L$ be the standard orthonormal basis of ${\mathbb{R}}^{2L}$ and defineThenThus $g$ is a positive-semidefinite kernel.

This covariance function is periodic with period $P>0$. Map time to the circle $u(t)=(\cos (2\pi t/P),\sin (2\pi t/P))$ and pull back a Gaussian kernel; since $\Vert u(t)-u(t^{\prime }){\Vert }^2=4{\sin }^2(\pi (t-t^{\prime })/P)$, this proves positive semidefiniteness. A Gaussian process with this kernel has $\mathrm{Var}(f(t+P)-f(t))=0$ and therefore repeats at every fixed phase. Replacing the inner $\pi$ by $2\pi$ changes the actual period to $P/2$.

A zero-mean Gaussian process need not have zero average in each realization. For the periodic covariance function, the constant Fourier component iswhere $I_0$ is a modified Bessel function. Replacing $f(t)$ by $f(t)-P^{-1}{\int }_0^{P}f(u)du$ gives a Gaussian process with kernel $k_P(t,t^{\prime })-c_0$. This is a positive-semidefinite kernel because it is a covariance after a linear transformation, and each sample has zero period average. The real integral representation of the modified Bessel function I0 yields the displayed constant.

For $x,z\in {\mathbb{R}}^d$, a polynomial kernel is the positive-semidefinite kernel $k(x,z)=(x^{T}z+c{)}^r$, where $r$ is a positive integer and $c\ge 0$. The case $c=0$ uses monomials of total degree $r$; $c>0$ also includes lower degrees. The feature map weights matter: listing each distinct monomial once with coefficient one generally gives a different inner product.^[ref]

For the homogeneous quadratic case in two dimensions, take the scaled quadratic feature mapThe inner product identity is explicit:The factor $\sqrt{2}$ accounts for the two equal cross terms in the square. Thus an inner product of three derived coordinates can be obtained from an inner product of the two input coordinates followed by squaring. The kernel trick uses this equality inside an algorithm; the feature map explains which nonlinear functions the algorithm can represent.

A symmetric function $k$ is a positive-semidefinite kernel when every finite Gram matrix $(k(x_i,x_j))$ is positive semidefinite.

The pointwise product of two positive-semidefinite kernels is positive semidefinite because its finite kernel matrix is the entrywise product of two positive-semidefinite matrices, to which the Schur product theorem applies.

Random Fourier features represent a translation-invariant positive-semidefinite kernel by sampling frequencies from its spectral distribution and using sine and cosine features. They replace an implicit infinite-dimensional feature map by a finite random approximation.

The representer theorem reduces regularized optimization over a reproducing-kernel Hilbert space to the span of kernel sections at the observed points.

A reproducing-kernel Hilbert space is a Hilbert space of functions in which point evaluation is continuous and satisfies $f(x)=⟨f,k(x,\cdot )⟩$.

If a stationary Gaussian process has covariance spectral density $\hat{K}$, its reproducing-kernel Hilbert space has normon the functions for which this integral is finite.

The squared-slack support vector machine uses constraints $y_i(w\cdot x_i+b)\ge 1-{\xi }_i$. With an unpenalized intercept, its Lagrangian dual problem maximizes ${\sum }_i{\alpha }_i-\frac{1}{2}{\sum }_{i,j}{\alpha }_i{\alpha }_j{y}_i{y}_{j}K(x_i,x_j)-\frac{1}{4C}{\sum }_i{\alpha }_i^2$ subject to ${\alpha }_i\ge 0$ and ${\sum }_i{\alpha }_i{y}_i=0$. The square adds a diagonal penalty in the dual rather than a finite upper box constraint. The Karush-Kuhn-Tucker conditions give $w={\sum }_i{\alpha }_i{y}_i\varphi (x_i)$ and ${\xi }_i={\alpha }_i/(2C)$ for a feature map realizing the positive-semidefinite kernel $K$.

For positive-semidefinite kernels $k_j$ with Reproducing kernel Hilbert spaces ${\mathcal{H}}_j$, the Reproducing kernel Hilbert space of $k={\sum }_j{k}_j$ consists of sums $f={\sum }_j{f}_j$, withThe minimum is attained uniquely. Indeed, in the Hilbert space direct sum $E={⨁}_j{\mathcal{H}}_j$, the tuples summing to zero form a closed subspace $N$: they are the intersection, over inputs $x$, of the kernels of the continuous maps $(f_j)\mapsto {\sum }_j{f}_j(x)$. Every affine fibre has a unique representative in $N^{\perp }$ by orthogonal decomposition by a closed subspace. The vector $(k_j(\cdot ,x){)}_j$ lies in $N^{\perp }$ and represents evaluation in the quotient; its evaluation at $y$ is ${\sum }_j{k}_j(y,x)$, identifying the positive-semidefinite kernel with $k$.