= Covariance function
{title2=$C(s,t)$}
For a real-valued <stochastic process> $(X_t)$ with finite <second moments>, write $m(t)=\mathbb E X_t$. Its covariance function is
$$
C(s,t)=\operatorname{Cov}(X_s,X_t)=\mathbb E[(X_s-m(s))(X_t-m(t))].
$$
The diagonal gives the <variance> at each time; off-diagonal values record linear dependence between times. For the independent <standard normal random variables> above, $C_X(m,n)=\mathbf1_{\{m=n\}}$, whereas the shared <Gaussian process> $Y_n=Z$ has $C_Y(m,n)=1$ for all $m,n$. For a fair <simple random walk on the integer line>, $C(m,n)=\min(m,n)$; <Brownian motion> has the continuous-time counterpart $C(s,t)=\min(s,t)$. These formulas quantify how much random history the observations share.
Every covariance function is a <positive-semidefinite kernel>. Indeed, for any real coefficients $a_1,\ldots,a_k$,
$$
\sum_{i,j=1}^k a_i a_j C(t_i,t_j)=\operatorname{Var}\!\left(\sum_{i=1}^k a_i X_{t_i}\right)\ge0.
$$
Thus a proposed covariance function must make every finite <covariance matrix> <positive semidefinite>. For a <Gaussian process>, the <mean> function and the covariance function determine every <finite-dimensional distribution>, including degenerate ones such as the shared process. Conversely, any real <positive-semidefinite kernel> satisfying $C(s,t)=C(t,s)$, together with a specified <mean> function, defines consistent <multivariate normal distributions>; the <Kolmogorov extension theorem> then constructs a <Gaussian process>. For a general <stochastic process>, <covariance> alone does not determine the full law. The exceptional completeness of this description is what makes <Gaussian process regression> possible with a mean and a covariance model.
Back to article page
