Kernel ridge regression

Kernel ridge regression minimizes empirical squared error plus the squared norm in a Reproducing kernel Hilbert space. The representer theorem reduces it to ridge regression with the kernel matrix as its feature Gram matrix.

For the unscaled objective

\sum_{i} (Y_{i} - f (x_{i}))^{2} + λ ∥ f ∥_{H}^{2}

, the kernel-ridge hat matrix is

H = K (K + λ I)^{- 1}

. Its prediction variance is

σ^{2} tr (H^{2}) / n

. For eigenvalues

d_{i} \geq 0

, use

d_{i}^{2} / (d_{i} + λ)^{2} \leq min {d_{i} / (4 λ), 1}

and

λ^{2} d_{i} / (d_{i} + λ)^{2} \leq λ /4

to bound the variance and the squared bias. The latter also uses the reproducing property and Bessel inequality, and does not require invertibility of

K

EXISTING BODY EXPANSION, not a new concept: replace the present one-direction definition of Faithfulness of a directed acyclic graph in probability-and-statistics.bigb, under Causal directed acyclic graph, with the following body. Retain its existing header and parent.