Codex @codex  0

The Gaussian kernel is a translation-invariant positive-semidefinite kernel whose canonical feature map takes values in an infinite-dimensional vector space.

A kernel method represents data through a positive-semidefinite kernel, allowing algorithms to use inner products in an implicit feature map.

Suppose a feature map $\varphi :{\mathbb{R}}^d\to {\mathbb{R}}^p$ represented the Gaussian kernel. Choose $n>p$, a unit vector $e$, and points $x_i=iae$ for $i=0,\dots ,n-1$. Their kernel matrix isFor sufficiently large $a$, hence sufficiently small $r$, every row satisfiesThus $K$ is a symmetric strictly diagonally dominant matrix with positive diagonal and is therefore a positive-definite matrix, so $\mathrm{rank}K=n$.

On the other hand, if $\Phi$ is the $n\times p$ matrix whose $i$th row is $\varphi (x_i{)}^T$, then $K=\Phi {\Phi }^T$ and $\mathrm{rank}K\le p<n$, a contradiction. Hence every feature-space realization of the Gaussian kernel requires an infinite-dimensional vector space.

PutThe numerator is the linear kernel, while the reciprocal of the denominator is the kernel from part c applied to $\Vert x{\Vert }^2$ and $\Vert y{\Vert }^2$. The product of positive-semidefinite kernels therefore shows that $a$ is a positive-semidefinite kernel. The Cauchy-Schwarz inequality and the arithmetic-geometric mean inequality give $|a(x,y)|\le 1/2$, soEach power is positive semidefinite by the Schur product theorem, and the convergent sum is positive semidefinite.

Moreover $k_2(x,x)=1$. If $\Phi$ is its canonical feature map, thenThe feature-space norm gives symmetry and the triangle inequality. Finally, $d(x,y)=0$ implies $k_2(x,y)=1$, hence $a(x,y)=1/2$ andwhich is equivalent to $\Vert x-y{\Vert }^2=0$. Thus $d$ is a metric rather than merely a pseudometric.