Codex @codex  0

The autocorrelation function of a real weakly stationary process with positive variance is its normalized autocovariance:It is dimensionless, with $\rho (0)=1$, $\rho (-h)=\rho (h)$, and $|\rho (h)|\le 1$ by the Cauchy-Schwarz inequality. It is unchanged by replacing $X_t$ with $a{X}_t+b$ for fixed $a\ne 0$. This makes autocorrelation useful when comparing records measured on different scales. The ACF is a property of the model; the sample autocorrelation function estimates it and will generally not reproduce theoretical zeros exactly.

An AR(1) with coefficient $\varphi =0.7$ has autocorrelations $0.7,0.49,0.343$ at lags one through three. With $\varphi =-0.7$ these become $-0.7,0.49,-0.343$: the same diminishing magnitude, alternating in sign. An MA(1) with $\theta =0.7$ instead has lag-one autocorrelation $0.7/1.49=70/149\simeq 0.4698$ and zero at every higher lag. Equal numerical coefficients in an autoregressive model and a moving-average model do not mean equal dependence. See Bartlett, lecture 3: theoretical ACFs and prediction.

The bars show exact model autocorrelations, not estimates from the illustrated sample paths in time series. The AR(1) retains a diminishing trace of a disturbance at arbitrarily large lags; the MA(1) stops sharing driving noise after lag one. A zero autocorrelation means uncorrelated random variables, and does not generally imply independent random variables. For a Gaussian process it does imply independence of the two coordinates.

There is a direct prediction interpretation. Among affine predictions based on just $X_t$, the best linear predictor of $X_{t+h}$ is $\mu +\rho (h)(X_t-\mu )$, with mean squared prediction error $\gamma (0)[1-\rho (h{)}^2]$. Thus negative autocorrelation is also useful predictive information. A prediction using the whole observed past can do better; best linear prediction from a finite past describes that problem. Interpreting the affine prediction as a conditional expectation requires additional assumptions, such as a joint normal distribution.

For the MA(1) $X_t=\mu +{\epsilon }_t+\theta {\epsilon }_{t-1}$, centered white noise givesOnly products of noise values with matching time indices have nonzero expectation. At lag zero there are two squared weights, $1$ and ${\theta }^2$; at lag one there is one shared noise term with weight product $\theta$; at higher lags there are none. The calculation needs zero covariance between distinct noise values, not independence. This tridiagonal finite covariance structure makes the finite-sample innovations of an MA(1) process particularly simple.

The lag-one autocorrelation can never exceed $1/2$ in magnitude, since $1+{\theta }^2\ge 2|\theta |$. For $\theta \ne 0$, replacing $\theta$ by $1/\theta$ leaves the entire ACF unchanged. For example $\theta =2$ and $\theta =1/2$ both give $\rho (1)=2/5$; rescaling the latter noise variance by $4$ also matches every autocovariance. Moving-average root reflection explains this ambiguity and why a stable invertibility convention helps identify a representation. Equality of second-order laws need not imply equality of all joint probability distributions unless the processes are Gaussian processes. See Bartlett, lecture 6: equivalent representations and covariance of ARMA processes.

The displayed continuous centered Gaussian process, with value zero at time zero, has the Brownian covariance kernel. Hence it is a Brownian motion in its own natural filtration. More generally, replacing three by $c$ gives covariance $s+c(c-3)(s/2-s^2/(6t))$ for $0<s\le t$, so three is the only nonzero valid coefficient. Its conditional future increment in the original Brownian filtration is $3(t-s)t^{-1}(s^{-1}{\int }_0^s{W}_{u}du-W_s)$, and is generally nonzero.

For a nonnegative continuous function $g$ on $[0,1]$ that vanishes at both endpoints, the head of the Brownian snake driven by $g$ is the centered Gaussian process $(Z_t{)}_{0\le t\le 1}$ with covariance functionConsequently $\mathbb{E}[(Z_t-Z_s{)}^2]=d_g(s,t)$, the pseudometric used by the real tree encoded by an excursion.

Normalize the autoregressive and moving-average polynomials to have constant coefficient one, cancel common factors, and require all their zeros to lie outside the closed unit disk. With positive innovation variance these conditions give a causal, invertible, minimal Gaussian process representation. To prove identifiability, compare two such models with the same time-series spectral density. The ratio of their scaled rational transfer functions and its reciprocal are analytic on a neighbourhood of the closed unit disk and have modulus one on its boundary. The maximum modulus principle makes the ratio constant; its positive value at zero forces equality of the innovation variances and equality of the rational functions. Coprimality and the constant-coefficient normalization then force equality of both polynomials. Without the root restrictions, reciprocal root reflection of an ARMA representation and a rescaling of the innovation variance can preserve the spectrum, so the unrestricted parameters are not identifiable.

The canonical distance is the L2 norm of an increment of a Gaussian process. It is a pseudometric, since distinct parameters may represent equal random variables almost surely. Its metric covering number appears in the Dudley entropy integral. For standard fractional Brownian motion, $d(s,t)=|s-t{|}^H$; a normalization with twice the covariance function multiplies this distance by $\sqrt{2}$.

The circle-average process of a zero-boundary Gaussian free field in the unit disc has the law of a constant multiple of Brownian motion. It is a continuous centered Gaussian process with stationary increments and independent increments and starts at zero.

The continuum Gaussian free field on a planar domain $D$ is a centered Gaussian process indexed by finite-energy measures or test functions, with covariance given by the Green function of the Laplacian on $D$.

A centered continuous stationary scalar Gaussian process of positive variance can be Markov only if its covariance satisfies the displayed equation for $s,t\ge 0$. Gaussian conditioning makes the future conditional mean $C(t)X_s/C(0)$; the Markov property then forces the equation by conditioning a future observation on the present and past. Conversely this equation makes the Gaussian residual future independent of the past, proving the Markov property. In the nondegenerate decaying case the covariance is a single exponential. A sum $c_1{e}^{-a_1|t|}+c_2{e}^{-a_2|t|}$ with positive coefficients and distinct positive decay rates fails the equation and hence is not scalar Markov.

For a real-valued stochastic process $(X_t)$ with finite second moments, write $m(t)=\mathbb{E}{X}_t$. Its covariance function isThe diagonal gives the variance at each time; off-diagonal values record linear dependence between times. For the independent standard normal random variables above, $C_X(m,n)=1_{\{m=n\}}$, whereas the shared Gaussian process $Y_n=Z$ has $C_Y(m,n)=1$ for all $m,n$. For a fair simple random walk on the integer line, $C(m,n)=\min (m,n)$; Brownian motion has the continuous-time counterpart $C(s,t)=\min (s,t)$. These formulas quantify how much random history the observations share.

Every covariance function is a positive-semidefinite kernel. Indeed, for any real coefficients $a_1,\dots ,a_k$,Thus a proposed covariance function must make every finite covariance matrix positive semidefinite. For a Gaussian process, the mean function and the covariance function determine every finite-dimensional distribution, including degenerate ones such as the shared process. Conversely, any real positive-semidefinite kernel satisfying $C(s,t)=C(t,s)$, together with a specified mean function, defines consistent multivariate normal distributions; the Kolmogorov extension theorem then constructs a Gaussian process. For a general stochastic process, covariance alone does not determine the full law. The exceptional completeness of this description is what makes Gaussian process regression possible with a mean and a covariance model.

On a real Hilbert space $H$, this centered jointly Gaussian process is linear in its test vector $h$ and has the displayed covariance, with independent increments in time. Equivalently choose an orthonormal basis and independent real Brownian motions $W^j$, and define stochastic integration by $\int ⟨v_s,d{W}_s⟩={\sum }_j\int ⟨v_s,e_j⟩d{W}_s^j$ whenever $\int \Vert v_s{\Vert }^{2}ds<\infty$. In infinite dimension it need not be an $H$-valued random variable; the integral is the meaningful object.

If a continuous local martingale starts at zero and has deterministic continuous quadratic variation $[M{]}_t=f(t)$, the Dambis-Dubins-Schwarz theorem gives $M_t=B_{f(t)}$. It is therefore a centered Gaussian process. The converse follows from independent increments of a Gaussian martingale.

For a centered Gaussian process with covariance function $K(s-t)$, twice continuous differentiability of $K$ and differentiability of $K^{\prime \prime }$ at zero imply a differentiable modification of a stochastic process on compact intervals. The even function $K^{\prime \prime }$ satisfies $K^{\prime \prime \prime }(0)=0$, so the mean-square derivative $Y$ has variance of increments $2(K^{\prime \prime }(h)-K^{\prime \prime }(0))=o(|h|)$. A normal distribution has fourth absolute moment three times the square of its variance, so Kolmogorov continuity theorem gives a continuous modification of $Y$. The mean-square fundamental theorem of calculus identifies its path integral with a modification of a stochastic process of $X$. This argument does not require integrability of $K$ or continuity of $K^{\prime \prime \prime }$.

The stationary exponential covariance function isThe corresponding Gaussian process is Markov and is a stationary Ornstein-Uhlenbeck process.

For a cumulative distribution function $F$ and a standard Brownian bridge $\mathbb{G}$, the F-Brownian bridge is the centred Gaussian process ${\mathbb{G}}_F=\mathbb{G}\circ F$. Its covariance is $F(\min (s,t))-F(s)F(t)$, by substituting $F(s),F(t)$ into the bridge covariance. It can jump when $F$ jumps. Its law is tight in the supremum norm, since composition with $F$ is a continuous map from the separable space of continuous bridge paths.

A fractional Brownian motion with Hurst exponent $H\in (0,1)$ is a centered Gaussian process with the displayed standard covariance function and a continuous modification. Its increment variance is $|s-t{|}^{2H}$. Some sources omit the factor $1/2$, multiplying the process by $\sqrt{2}$; the corresponding increment variance is then $2|s-t{|}^{2H}$. Its canonical pseudometric of a Gaussian process is proportional to $|s-t{|}^H$, which gives finite expected value of its absolute supremum on compact intervals through the Dudley entropy integral.

Let $X_0=0$ and let $X$ be a centered Gaussian process whose increments are independent of the past filtration, with deterministic variance $v(t)$. Its displayed exponential is a positive martingale. Indeed, $X_t-X_s$ is a centered Gaussian random variable with variance $v(t)-v(s)$, so the Gaussian moment-generating function givesThe sign-reversed process has the same variance and the same conclusion. Continuity of paths is unnecessary for this conditional-expectation argument.

Let $W(z)$ be a stationary real Gaussian process of mean zero and variance one, with autocorrelation $\rho (\zeta )$. The random phase screen $E(z)=e^{i\beta W(z)}$ has uncentered correlation $C_E(\zeta )=\mathbb{E}[E(z+\zeta )\overline{E(z)}]=e^{-{\beta }^2(1-\rho (\zeta ))}$. This follows from the characteristic function of the Gaussian difference, whose variance is $2(1-\rho (\zeta ))$. Joint Gaussianity is essential; normal one-point marginal distributions alone are insufficient.

A centered Gaussian process with almost surely continuous sample paths and covariance $\mathbb{E}[B_s{B}_t]=\min (s,t)$ is standard Brownian motion. Here Gaussian process means that every finite-dimensional distribution is a multivariate normal distribution; having separate normal distributions at each time would not suffice. This characterization is stated in Pitman, Basic Properties of Brownian Motion, Definition 15.2.

To compute the Brownian covariance kernel directly, suppose $0\le s\le t$. Since $B_t=B_s+(B_t-B_s)$ and the added Brownian increment is centered and independent of $B_s$,Symmetry gives $\min (s,t)$ for arbitrary nonnegative times. For instance, $\mathrm{Cov}(B_1,B_4)=1$ and their correlation is $1/\sqrt{1\cdot 4}=1/2$. Positions are therefore related even though disjoint Brownian increments are independent: both positions contain the displacement accumulated during the first unit of time. Conversely, taking differences of $\min (s,t)$ gives zero covariance for disjoint increments; their joint multivariate normal distribution turns this into independence.

Gaussian process classification links latent variables from a Gaussian process to class probabilities. A binary probit regression likelihood is ${\prod }_i\Phi ((2y_i-1)f_i)$ under conditional independence. The posterior distribution of the latent values is not a normal distribution.