Covariance operator ($C_X$)

For a square-integrable centered Hilbert-space-valued random variable $X$, its covariance operator isIt is a positive self-adjoint trace-class operator, and $\mathrm{tr}{C}_X=\mathbb{E}\Vert X{\Vert }^2$.

When a covariance operator on a function space is an integral operator, its covariance kernel satisfies $(C_{X}f)(s)=\int c_X(s,t)f(t)dt$. For a centered process, $c_X(s,t)=\mathbb{E}[X(s)X(t)]$ whenever point evaluation is meaningful.

The covariance kernel of standard Brownian motion on $[0,1]$ is $c(s,t)=\min (s,t)$. Its integral operator has eigenfunctions $\sqrt{2}\sin ((k-\frac{1}{2})\pi t)$ and eigenvalues $((k-\frac{1}{2})\pi {)}^{-2}$.

For a complete orthonormal basis of $L^2({\mathbb{R}}_{+})$, the Parseval identity for a Hilbertian basis identifies the sum of the feature products with $⟨1_{[0,s]},1_{[0,t]}⟩=\min (s,t)$. The Cauchy-Schwarz inequality gives absolute convergence.

The covariance kernel of a standard Brownian bridge on $[0,1]$ is $c(s,t)=\min (s,t)-st$. Its integral operator has eigenfunctions $\sqrt{2}\sin (k\pi t)$ and eigenvalues $(k\pi {)}^{-2}$ for $k\ge 1$.

For observations $X_1,\dots ,X_n$ and a chosen center $m$, the empirical covariance operator is ${\hat{C}}_m=n^{-1}{\sum }_i(X_i-m)\otimes (X_i-m)$. Its expectation equals the population covariance plus the rank-one operator formed from the centering error.

A pooled covariance operator averages the within-group empirical covariance operators from several samples that are assumed to share one population covariance. Centering within each group prevents differences between group means from contaminating the covariance estimate.