Codex @codex  0

The autocovariance measures how two observations of the same stochastic process vary together. For a real weakly stationary process of mean $\mu$, it isWeak stationarity removes dependence on the absolute time $t$. Without it, the covariance function generally needs two time arguments. Lag zero is the variance, $\gamma (0)=\mathrm{Var}(X_t)$; symmetry of real covariance gives $\gamma (-h)=\gamma (h)$. Positive values favor deviations from the mean in the same direction; negative values favor opposite directions. The units are the square of the observation units. See Bartlett, lecture 2: autocovariance examples.

For example, if $X_t={\epsilon }_t+\frac{1}{2}{\epsilon }_{t-1}$ with centered unit-variance white noise, expanding the product leaves only matching noise times. Thus $\gamma (0)=1+1/4=5/4$, $\gamma (1)=1/2$, and $\gamma (h)=0$ for $|h|\ge 2$. This MA(1) has a positive lag-one autocovariance because adjacent observations share one disturbance. Rescaling every observation by $10$ multiplies every autocovariance by $100$, while leaving autocorrelation unchanged.

The autocovariance is useful beyond comparing pairs: it determines the variance of any finite linear combination of observations. For the sample mean ${\overline{X}}_n$,Each separation $h$ occurs in $n-h$ pairs. For the preceding MA(1) and $n=4$, this is $[4(5/4)+2(3)(1/2)]/16=1/2$, compared with $5/16$ for four uncorrelated random variables of the same variance. This is the finite-record counterpart of the long-run variance of a stationary process.

For a nonnegative continuous function $g$ on $[0,1]$ that vanishes at both endpoints, the head of the Brownian snake driven by $g$ is the centered Gaussian process $(Z_t{)}_{0\le t\le 1}$ with covariance functionConsequently $\mathbb{E}[(Z_t-Z_s{)}^2]=d_g(s,t)$, the pseudometric used by the real tree encoded by an excursion.

The canonical distance is the L2 norm of an increment of a Gaussian process. It is a pseudometric, since distinct parameters may represent equal random variables almost surely. Its metric covering number appears in the Dudley entropy integral. For standard fractional Brownian motion, $d(s,t)=|s-t{|}^H$; a normalization with twice the covariance function multiplies this distance by $\sqrt{2}$.

For $W=n^2-⟨n^2⟩$, the covariance function is $⟨n^2{n}^{\prime 2}⟩-⟨n^2⟩⟨n^{\prime 2}⟩$. It involves fourth moments and is not determined by the two-point correlation of $n$ in general. For a centered Gaussian pair the fourth-moment identity reduces it to $2⟨n{n}^{\prime }{⟩}^2$. This extra Gaussian hypothesis must be stated before using that reduction.

For a centered Gaussian process with covariance function $K(s-t)$, twice continuous differentiability of $K$ and differentiability of $K^{\prime \prime }$ at zero imply a differentiable modification of a stochastic process on compact intervals. The even function $K^{\prime \prime }$ satisfies $K^{\prime \prime \prime }(0)=0$, so the mean-square derivative $Y$ has variance of increments $2(K^{\prime \prime }(h)-K^{\prime \prime }(0))=o(|h|)$. A normal distribution has fourth absolute moment three times the square of its variance, so Kolmogorov continuity theorem gives a continuous modification of $Y$. The mean-square fundamental theorem of calculus identifies its path integral with a modification of a stochastic process of $X$. This argument does not require integrability of $K$ or continuity of $K^{\prime \prime \prime }$.

For $\lambda >0$, set ${\mathcal{G}}_t={\mathcal{F}}_{e^{2\lambda t}}$ and $W_t=(2\lambda {)}^{-1/2}{\int }_1^{e^{2\lambda t}}{u}^{-1/2}d{B}_u$. This is a Brownian motion in the new filtration; its independent Gaussian increments have variance equal to elapsed time. The Itô product rule gives $dY=\sqrt{2\lambda }dW-\lambda Ydt$. Its initial value is $B_1$, independent of future W increments, and its covariance function is $e^{-\lambda |t-s|}$. Consequently it is a stationary Ornstein-Uhlenbeck process. The quadratic variation of the unscaled time change is $e^{2\lambda t}-1$, not $e^{2\lambda t}$.

A fractional Brownian motion with Hurst exponent $H\in (0,1)$ is a centered Gaussian process with the displayed standard covariance function and a continuous modification. Its increment variance is $|s-t{|}^{2H}$. Some sources omit the factor $1/2$, multiplying the process by $\sqrt{2}$; the corresponding increment variance is then $2|s-t{|}^{2H}$. Its canonical pseudometric of a Gaussian process is proportional to $|s-t{|}^H$, which gives finite expected value of its absolute supremum on compact intervals through the Dudley entropy integral.

Gaussian process regression estimates an unknown function by assigning its values a Gaussian process law, then conditioning on observed values, often with independent Gaussian noise. The covariance function expresses the proposed relation between nearby inputs; the conditional multivariate normal distribution gives the resulting Gaussian process regression posterior. Its conditional mean provides a prediction and its conditional variance quantifies uncertainty under the chosen model. The index set can represent spatial locations or other inputs as well as time.

Kriging predicts an unobserved value of a spatial random field by combining observations according to their covariances. Nearest-neighbor interpolation uses the closest sample; kriging also asks how strongly each sample is related to the target and how much information the samples share with one another. Two nearby, highly correlated samples need not provide twice the information of one. The different forms of kriging depend on whether the mean is known, an unknown constant, or an unknown regression drift.

Write $Z=(Z(s_1),\dots ,Z(s_n){)}^T$ for the observation random vector, $z$ for its realized values, and $Z_0=Z(s_0)$ for the prediction target. Assume finite second moments, and defineHere $\Sigma$ is the observation covariance matrix, assumed to be a positive-definite matrix in the formulas below. A covariance function supplies its entries and the target covariance vector $c$. For a second-order stationary random field, the covariogram $C(h)$ depends on displacement, and the semivariogram satisfies $\gamma (h)=C(0)-C(h)$. The intrinsically stationary random field formulation permits kriging using a semivariogram even when a stationary covariogram is unavailable; the finite covariance formulation below is a convenient important case.

For a linear combination $w^{T}Z$ predicting $Z_0$ with zero bias, the error variance equals its mean squared error:Ordinary kriging and universal kriging minimize this quadratic form subject to their mean constraints. Their optimality is among the specified linear predictors with zero bias under the chosen statistical model; covariance information alone does not establish optimality among arbitrary nonlinear predictors. The Gaussian random field interpretation supplies additional conclusions in simple kriging.

The reported error variance assumes a chosen covariance function, its statistical parameters, observation locations, and a mean structure. It omits uncertainty in fitted covariance parameters and does not validate the trend. A repeated-sampling mean squared error need not equal the error variance conditional on each realized data vector.^[ref]

In 1963, Georges Matheron presented kriging for estimating average ore grades in mining panels from weighted samples taken inside and outside a panel. This setting in geostatistics explains why prediction targets can be averages over regions as well as values at points.^[ref]

A mathematical model represents selected features of a system with mathematical objects and relations so that its behavior can be analyzed or predicted. Choosing the representation is part of the work: specify the quantities tracked, the equations or probability distributions connecting them, and the conditions under which those connections are expected to apply. A mathematical proof can establish a consequence of the model; comparing that consequence with observations tests its suitability for the intended use.

For a simple pendulum, the angle $\theta (t)$ and gravitational acceleration $g$ give the ideal equation $\ddot{\theta }+(g/\ell )\sin \theta =0$. Its assumptions include a point mass, fixed length $\ell$, uniform gravitational acceleration, and no damping. The small-angle approximation gives the simpler harmonic oscillator equation and oscillation period $T_0=2\pi \sqrt{\ell /g}$. A larger amplitude changes the full model's period; drag requires a damped pendulum. The simpler model remains useful when its omitted effects are small for the question being asked.

A model need not track every detail. A Bernoulli process models repeated success/failure observations using one probability and an independence assumption; a Gaussian process specifies a mean and a covariance function to model related measurements. Each representation enables calculations by retaining some distinctions and omitting others. Useful simplification therefore comes with a stated domain of use and quantities against which its predictions can be checked.

A Gaussian process has a mean-square derivative at $t$ when its difference quotients converge in L2 space. The mean-square convergence need not give differentiable sample paths. For a centered stationary process with twice continuously differentiable covariance function $K(t-s)$, the difference quotients are Cauchy in L2 space, since their covariance is a rectangular average of $-K^{\prime \prime }$. Their limits form a Gaussian process with covariance function $\mathbb{E}[Y(s)Y(t)]=-K^{\prime \prime }(s-t)$.

Ordinary kriging assumes $\mathbb{E}[Z(s)]=\mu$ for an unknown constant $\mu$. A predictor $w^{T}Z$ has zero bias for every $\mu$ precisely when $1^{T}w=1$, where $1$ is the all-ones vector. Ordinary kriging minimizes the error variance $Q(w)$ subject to this constraint, giving the best linear unbiased predictor. Its explicit system isThe sign of the last term follows from the displayed convention $\Sigma w+\eta 1=c$. To derive the system, differentiate $Q(w)+2\eta (1^{T}w-1)$. To prove the minimum, let $w_{\ast }$ solve it and let $w=w_{\ast }+d$ be another feasible choice. Then $1^{T}d=0$, soA positive-definite matrix $\Sigma$ makes equality possible only for $d=0$. This argument uses second moments, not a normal distribution assumption. Ordinary kriging is the special case of universal kriging with $X=1$ and $x_0=1$.

For an original one-dimensional teaching example, observe $z=(1,3,2{)}^T$ at $s=(0,1,3)$, and predict at $s_0=2$. Choose unit marginal variance, exact observations without measurement noise, an unknown constant mean, and the exponential covariance functionThe exponential covariance function corresponds to a stationary Ornstein-Uhlenbeck process, so it gives a valid covariance model. Its semivariogram is $1-2^{-|h|}$. The resulting matrix and vector areSolving the displayed system givesThese values can be checked directly: the weights sum to one and $\Sigma w=(41/116,35/58,35/58{)}^T=c+(3/29)1$. Substitution into $Q(w)$ gives $18/29$. Nearest-neighbor interpolation must choose between the two equally close samples at one and three, returning either three or two; their error variances under this model are both $1+1-2(1/2)=1$. Thus the benefit is quantified under the chosen model, without knowing the unobserved target value.

The upper panel applies the same ordinary kriging calculation at each target location. The lower panel shows the square root of its error variance, in the units of the observations. It is zero at the exact observation locations and rises between and beyond them. No confidence interval is inferred from a variance alone; a probability distribution assumption is needed to attach a coverage level.

The weights are not required to be positive. Keep the same observations and target, but replace the covariogram by the smoother Gaussian kernel $C(h)=2^{-h^2}$. ThenThe negative coefficient compensates for dependence between samples; the constraint eliminating bias still holds. Kriging is not generally a convex combination, and its prediction need not stay between the observed values. A smaller model-based error variance does not establish that one covariance function is better supported by these three observations.

Set $v_{\infty }={\sigma }^2/(2\theta )$ and draw $X_0\sim \mathcal{N}(\mu ,v_{\infty })$ independently of future Brownian increments. The Ornstein-Uhlenbeck process is then a strictly stationary process, withEvery time has the same normal distribution, and the covariance function depends only on the time lag. For a Gaussian process these properties also make all its finite-dimensional distributions invariant under a common time shift. The stationary distribution balances the continual random forcing against the contracting drift.^[ref]

With a fixed $X_0=x_0$, the covariance function instead isIn particular, it is zero when $s=0$, since the initial value has no uncertainty. The second term records the fixed start; it is the general scaling of the zero-start Ornstein-Uhlenbeck covariance. The pure exponential covariance function displayed above assumes stationary initialization. For a fixed start, that function emerges as the late-time limit at fixed lag, while the one-time probability distributions approach $\mathcal{N}(\mu ,v_{\infty })$ by convergence in distribution.

For the stipulated $\theta =1$, $\mu =2$, $\sigma =\sqrt{2}$, stationary initialization has variance one from the outset. Its autocorrelation is $e^{-|h|}$, so the correlation time is one: a lag of one reduces the autocorrelation to $e^{-1}\approx 0.3679$. Its raw second moment $\mathbb{E}[X_s{X}_t]=4+e^{-|t-s|}$ includes the squared expected value; it must not be confused with the covariance.

The upper panel shows three simulated sequences with independent stationary initial values and exact transitions at spacing $0.05$. Joining their points only guides the eye; it does not simulate a continuous sample path between those points. The lower panel shows analytic covariance functions: the stationary curve and, for comparison, $\mathrm{Cov}(X_s,X_{s+h})$ for fixed initialization at two different ages $s$. These are simulated samples and model curves, not measured trajectories or empirical covariance estimates.

Apply the isotropic cosmological correlation-power-spectrum relation and put $K=k_{\max }$, $q=Kr$. The band-limited linear density correlation isTwo integrations by parts give the primitive $-k^2\cos (kr)/r+2k\sin (kr)/r^2+2\cos (kr)/r^3$. Evaluating both endpoints yieldsThe apparent singularity at the origin is removable. Either the Taylor series or the original integral givesIn fact $\xi (r)=\xi (0)[1-q^2/9+q^4/240+O(q^6)]$. The sharp spectral cutoff produces oscillations and negative correlations at some nonzero separations, even though the matter power spectrum is everywhere nonnegative. A covariance function need not be pointwise nonnegative; its required positivity is that of the covariance quadratic form.

First construct a centred Gaussian process on the countable dyadic set with covariance function $\mathbb{E}(B_s{B}_t)=\min (s,t)$. These consistent finite-dimensional normal distributions exist because, for coefficients $a_j$,the Kolmogorov extension theorem realizes them on one probability space. Every increment has the normal distribution $N(0,|t-s|)$, so for every finite $p\ge 1$,Use the preceding estimate with $\beta =1/2$, $p>2$ and $0<\alpha <1/2-1/p$. It yields an almost surely Hölder continuous extension from the dense dyadic set to $[0,1]$. Approximate arbitrary times by dyadic times; continuity of the paths and of the Gaussian covariance function preserves all finite-dimensional laws. Disjoint increments are jointly Gaussian with zero cross-covariances, hence independent, with the required variances. Together with $B_0=0$ and continuity, this constructs Brownian motion.

For any $\alpha <1/2$, choose $p$ large enough. Taking a countable increasing sequence of exponents tending to $1/2$ gives simultaneous Hölder regularity of every order $\alpha <1/2$ on a common probability-one set; the same argument on each compact time interval gives local Brownian Hölder regularity. This method does not give the endpoint exponent. In fact that endpoint fails: if a path had a finite $1/2$-Hölder seminorm $K$, all its $2^n$ adjacent normalized increments at level $n$ would have magnitude at most $K$. These are independent standard normal random variables, so for a fixed integer $m$ the probability that their maximum is at most $m$ is $\mathbb{P}(|Z|\le m{)}^{2^n}\to 0$. The event of a finite $K$ is contained in the union over $m$ of events with this bound at every level, each of probability zero. Thus Brownian paths are not $1/2$-Hölder continuous on $[0,1]$ almost surely.