Past exam of the mathematics course of the University of Cambridge / 2021 / iii / Paper 205 / 4 / b

Solution

The kernel ridge regression estimator minimizes

\sum_{i = 1}^{n} {y_{i} - f (x_{i})}^{2} + λ ∥ f ∥_{H}^{2} .

By the representer theorem, its fitted-value vector is

f = K (K + λ I)^{- 1} y .

f

for the true value vector, the variance contribution to

E ∥ f - f ∥_{2}^{2}

σ^{2} tr {K^{2} (K + λ I)^{- 2}} = σ^{2} \sum_{i = 1}^{n} \frac{d _{i}^{2}}{( d _{i} + λ ) ^{2}} .

The squared bias is

∥ λ (K + λ I)^{- 1} f ∥_{2}^{2}

K

, the scalar inequality

\frac{λ ^{2}}{( d + λ ) ^{2}} \leq \frac{λ}{4 d},

which is equivalent to

4 d λ \leq (d + λ)^{2}

, yields

E i = 1 \sum n {f^{0} (x_{i}) - f_{λ} (x_{i})}^{2} \leq σ^{2} i = 1 \sum n \frac{d _{i}^{2}}{( d _{i} + λ ) ^{2}} + \frac{λ}{4} f^{T} K^{- 1} f .

For

∥ f ∥_{2} = 1

, only the last term varies. The Rayleigh quotient of

K^{- 1}

is maximized by a unit eigenvector associated with its largest eigenvalue

1/ d_{n}

, equivalently an eigenvector of

K

associated with

d_{n}

There are no discussions about this article yet.

There are currently no matching articles.