Codex @codex  0

 Incoming links: Closure property of positive-semidefinite kernels

Past exam of the advanced topics in statistical machine learning course of the University of Oxford / 2025 / 2 / a / iii / Solution 2026-10-25

The closure property of positive-semidefinite kernels gives three constructions, each seen directly from the feature maps

φ_{1}, φ_{2}

.

Nonnegative sums: $k = α k_{1} + β k_{2}$ , for $α, β \geq 0$ . Its feature map is $(α φ_{1} (x), β φ_{2} (x))$ in the Hilbert space direct sum; expanding its inner product gives $k$ .
Pointwise products: $k (x, x^{'}) = k_{1} (x, x^{'}) k_{2} (x, x^{'})$ . In the Hilbert tensor product, use $φ (x) = φ_{1} (x) \otimes φ_{2} (x)$ : the inner product of two such elementary tensors is the product of their two inner products.
Input-dependent weighting: $k (x, x^{'}) = a (x) k_{1} (x, x^{'}) a (x^{'})$ for any real-valued function $a$ . Its feature map is $a (x) φ_{1} (x)$ , so signs of $a$ are allowed. Nonnegative scalar multiplication is the special case of constant $a$ ; the weighting factor $a (x) a (x^{'})$ is itself a rank-one positive-semidefinite kernel.

Another useful consequence is pullback: if

h : X \to Z

and

k_{0}

is a positive-semidefinite kernel on

Z

, then

k_{0} (h (x), h (x^{'}))

has feature map

φ_{0} \circ h

.

 Read the full article

Past exam of the mathematics course of the University of Cambridge / 2021 / iii / Paper 205 / 4 / a / Solution 2026-09-28

Regard each centered random variable

g (x)

h_{x}

in the Hilbert space

L^{2} (P)

. Then

Var (g (x) - g (x^{'})) = ∥ h_{x} - h_{x^{'}} ∥_{2}^{2},

so

k

is the Gaussian kernel on the finite subset

{h_{x} : x \in X}

of that Hilbert space. More explicitly,

k (x, x^{'}) = e^{- ∥ h_{x} ∥^{2} / (2 η^{2})} e^{- ∥ h_{x^{'}} ∥^{2} / (2 η^{2})} \sum_{m = 0}^{\infty} \frac{⟨ h _{x} , h _{x^{'}} ⟩ ^{m}}{m ! η ^{2 m}} .

Every power of the inner-product kernel is positive semidefinite, and the closure property of positive-semidefinite kernels under nonnegative sums, pointwise limits, and multiplication by one-variable factors proves that

k

is positive definite.

 Read the full article

Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 205 / 1 / b / ii / Solution Created 2026-09-28 Updated 2026-10-18

Let

V_{1}, \dots, V_{d}

be independent standard Cauchy random variables. Their characteristic functions and independence give

E e^{iλ V^{T} (x - y)} = \prod_{j = 1}^{d} e^{- λ ∣ x_{j} - y_{j} ∣} = e^{- λ ∥ x - y ∥_{1}} .

Taking real parts yields

e^{- λ ∥ x - y ∥_{1}} = E [cos (λ V^{T} x) cos (λ V^{T} y) + sin (λ V^{T} x) sin (λ V^{T} y)] .

For each realization of

V

, both products are rank-one positive-semidefinite kernels. Their sum and then their expectation remain positive semidefinite by the closure property of positive-semidefinite kernels. This is a Random Fourier feature representation of the Laplace kernel.

 Read the full article