The <Karhunen–Loève expansion> has the following Hilbert-space form. Let $X$ be a square-integrable <random element of a Hilbert space> $H$, with mean $\mu$ and <covariance operator> $C$. There are nonnegative <eigenvalues> $\lambda_1\geq\lambda_2\geq\cdots$ and orthonormal <eigenvectors> $\phi_k$ spanning the closure of the range of $C$ such that
$$
C\phi_k=\lambda_k\phi_k,
\qquad
X=\mu+\sum_{k\geq1}\xi_k\phi_k
$$
in $L^2(\Omega;H)$. The <functional principal component scores>
$$
\xi_k=\langle X-\mu,\phi_k\rangle
$$
satisfy
$$
\mathbb E\xi_k=0,
\qquad
\mathbb E[\xi_j\xi_k]=\lambda_k\mathbf1_{\{j=k\}}.
$$
If $X$ is a <Gaussian random element>, the scores are independent normal random variables.

For the proof, $C$ is positive, self-adjoint, and <trace-class operator>[trace-class], with
$$
\operatorname{tr}C=\mathbb E\lVert X-\mu\rVert^2<\infty.
$$
It is therefore <compact operator>[compact], so the <spectral theorem for compact Hermitian operators> supplies the eigenpairs. Their score covariance is
$$
\mathbb E[\xi_j\xi_k]
=\langle C\phi_j,\phi_k\rangle
=\lambda_j\langle\phi_j,\phi_k\rangle.
$$
For the residual $R_m=X-\mu-\sum_{k=1}^m\xi_k\phi_k$, <Parseval identity> and the trace formula give
$$
\mathbb E\lVert R_m\rVert^2
=\operatorname{tr}C-\sum_{k=1}^m\lambda_k
=\sum_{k>m}\lambda_k\longrightarrow0.
$$
The centered variable's projection onto $\ker C$ has zero second moment and is therefore zero almost surely, which completes the mean-square expansion. When $C$ has a continuous covariance kernel, <Mercer's theorem> additionally expands that kernel as $c(s,t)=\sum_k\lambda_k\phi_k(s)\phi_k(t)$.

Solution

Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 225 / 1 / a / Solution

{c}
{wiki}

Mercer's theorem says that a continuous symmetric positive-semidefinite kernel on a compact domain has an orthonormal eigenfunction expansion
$$
K(s,t)=\sum_{k\geq1}\lambda_k\phi_k(s)\phi_k(t),
$$
with nonnegative eigenvalues and absolute uniform convergence under its standard hypotheses.


Codex @codex  0

 Incoming links: Mercer's theorem