Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 205 / 1

A symmetric function $k:\mathcal{X}\times \mathcal{X}\to \mathbb{R}$ is a positive-semidefinite kernel when, for every $m$, every $x_1,\dots ,x_m\in \mathcal{X}$, and every $c\in {\mathbb{R}}^m$,Equivalently, every associated kernel matrix is a positive semidefinite matrix.

A Reproducing-kernel Hilbert space $\mathcal{H}$ on $\mathcal{X}$ is a Hilbert space of real-valued functions such that every point-evaluation map $f\mapsto f(x)$ is continuous. The Riesz representation theorem then gives a function $k(x,\cdot )\in \mathcal{H}$ satisfying the reproducing propertyIts reproducing kernel is $k(x,y)=⟨k(x,\cdot ),k(y,\cdot ){⟩}_{\mathcal{H}}$.

Suppose a feature map $\varphi :{\mathbb{R}}^d\to {\mathbb{R}}^p$ represented the Gaussian kernel. Choose $n>p$, a unit vector $e$, and points $x_i=iae$ for $i=0,\dots ,n-1$. Their kernel matrix isFor sufficiently large $a$, hence sufficiently small $r$, every row satisfiesThus $K$ is a symmetric strictly diagonally dominant matrix with positive diagonal and is therefore a positive-definite matrix, so $\mathrm{rank}K=n$.

On the other hand, if $\Phi$ is the $n\times p$ matrix whose $i$th row is $\varphi (x_i{)}^T$, then $K=\Phi {\Phi }^T$ and $\mathrm{rank}K\le p<n$, a contradiction. Hence every feature-space realization of the Gaussian kernel requires an infinite-dimensional vector space.

For $x,y>0$, the Laplace transform identityexhibits $k_1$ as an inner product of the feature functions $t\mapsto e^{-tx}$ in $L^2(0,\infty )$. More explicitly, for any real $c_i$ and positive $x_i$,Therefore $k_1$ is a positive-semidefinite kernel.

PutThe numerator is the linear kernel, while the reciprocal of the denominator is the kernel from part c applied to $\Vert x{\Vert }^2$ and $\Vert y{\Vert }^2$. The product of positive-semidefinite kernels therefore shows that $a$ is a positive-semidefinite kernel. The Cauchy-Schwarz inequality and the arithmetic-geometric mean inequality give $|a(x,y)|\le 1/2$, soEach power is positive semidefinite by the Schur product theorem, and the convergent sum is positive semidefinite.

Moreover $k_2(x,x)=1$. If $\Phi$ is its canonical feature map, thenThe feature-space norm gives symmetry and the triangle inequality. Finally, $d(x,y)=0$ implies $k_2(x,y)=1$, hence $a(x,y)=1/2$ andwhich is equivalent to $\Vert x-y{\Vert }^2=0$. Thus $d$ is a metric rather than merely a pseudometric.