Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 225 / 1

The Karhunen–Loève expansion has the following Hilbert-space form. Let $X$ be a square-integrable random element of a Hilbert space $H$, with mean $\mu$ and covariance operator $C$. There are nonnegative eigenvalues ${\lambda }_1\ge {\lambda }_2\ge \cdots$ and orthonormal eigenvectors ${\varphi }_k$ spanning the closure of the range of $C$ such thatin $L^2(\Omega ;H)$. The functional principal component scoressatisfyIf $X$ is a Gaussian random element, the scores are independent normal random variables.

For the proof, $C$ is positive, self-adjoint, and trace-class, withIt is therefore compact, so the spectral theorem for compact Hermitian operators supplies the eigenpairs. Their score covariance isFor the residual $R_m=X-\mu -{\sum }_{k=1}^m{\xi }_k{\varphi }_k$, Parseval identity and the trace formula giveThe centered variable's projection onto $\ker C$ has zero second moment and is therefore zero almost surely, which completes the mean-square expansion. When $C$ has a continuous covariance kernel, Mercer's theorem additionally expands that kernel as $c(s,t)={\sum }_k{\lambda }_k{\varphi }_k(s){\varphi }_k(t)$.

The stated kernel is the Brownian bridge covariance kernel. Its eigenvalue equation isThe right-hand side vanishes at $s=0$ and $s=1$, and differentiating it twice givesThus the normalized eigenfunctions and eigenvalues areThe Karhunen–Loève expansion is consequentlywith convergence in $L^2(\Omega ;L^2[0,1])$, where $\mathbb{E}{Z}_k=0$ and $\mathbb{E}[Z_j{Z}_k]=1_{\{j=k\}}$. Covariance alone does not imply that the $Z_k$ are independent or normal; they are independent standard normal variables when $X$ is Gaussian.