Aki Vehtari 2026-11-05
Aki Vehtari develops computational Bayesian statistics, including probabilistic programming, model diagnostics, model assessment and Gaussian processes. His author research page describes these as connected parts of a Bayesian workflow: specify a model, compute its implications, check the computation and examine predictive adequacy.[ref]
He coauthored Bayesian Data Analysis by Andrew Gelman and others. A posterior distribution summarizes uncertainty conditional on the model and data; checking whether that model reproduces important observations is a further task. This distinction connects Bayesian inference with posterior predictive checks, rather than treating successful computation as sufficient evidence that the model is suitable.[ref]
Autocorrelation Created 2026-09-28 Updated 2026-10-14
The autocorrelation function of a real weakly stationary process with positive variance is its normalized autocovariance:
It is dimensionless, with , , and by the Cauchy-Schwarz inequality. It is unchanged by replacing with for fixed . This makes autocorrelation useful when comparing records measured on different scales. The ACF is a property of the model; the sample autocorrelation function estimates it and will generally not reproduce theoretical zeros exactly.
An AR(1) with coefficient has autocorrelations at lags one through three. With these become : the same diminishing magnitude, alternating in sign. An MA(1) with instead has lag-one autocorrelation and zero at every higher lag. Equal numerical coefficients in an autoregressive model and a moving-average model do not mean equal dependence. See Bartlett, lecture 3: theoretical ACFs and prediction.
Figure 1.
Theoretical autocorrelation: AR(1) with coefficients 0.7 and -0.7, compared with MA(1) with coefficient 0.7
. Details.
The bars show exact model autocorrelations, not estimates from the illustrated sample paths in time series. The AR(1) retains a diminishing trace of a disturbance at arbitrarily large lags; the MA(1) stops sharing driving noise after lag one. A zero autocorrelation means uncorrelated random variables, and does not generally imply independent random variables. For a Gaussian process it does imply independence of the two coordinates.
There is a direct prediction interpretation. Among affine predictions based on just , the best linear predictor of is , with mean squared prediction error . Thus negative autocorrelation is also useful predictive information. A prediction using the whole observed past can do better; best linear prediction from a finite past describes that problem. Interpreting the affine prediction as a conditional expectation requires additional assumptions, such as a joint normal distribution.
Autocovariance of an MA(1) process Created 2026-10-07 Updated 2026-10-14
For the MA(1) , centered white noise gives
Only products of noise values with matching time indices have nonzero expectation. At lag zero there are two squared weights, and ; at lag one there is one shared noise term with weight product ; at higher lags there are none. The calculation needs zero covariance between distinct noise values, not independence. This tridiagonal finite covariance structure makes the finite-sample innovations of an MA(1) process particularly simple.
The lag-one autocorrelation can never exceed in magnitude, since . For , replacing by leaves the entire ACF unchanged. For example and both give ; rescaling the latter noise variance by also matches every autocovariance. Moving-average root reflection explains this ambiguity and why a stable invertibility convention helps identify a representation. Equality of second-order laws need not imply equality of all joint probability distributions unless the processes are Gaussian processes. See Bartlett, lecture 6: equivalent representations and covariance of ARMA processes.
Borell-TIS inequality Created 2026-10-03 Updated 2026-11-05
If a centered separable Gaussian process has finite expected supremum and , then
The displayed continuous centered Gaussian process, with value zero at time zero, has the Brownian covariance kernel. Hence it is a Brownian motion in its own natural filtration. More generally, replacing three by gives covariance for , so three is the only nonzero valid coefficient. Its conditional future increment in the original Brownian filtration is , and is generally nonzero.
Brownian snake Created 2026-09-28 Updated 2026-11-05
For a nonnegative continuous function on that vanishes at both endpoints, the head of the Brownian snake driven by is the centered Gaussian process with covariance function
Consequently , the pseudometric used by the real tree encoded by an excursion.
Normalize the autoregressive and moving-average polynomials to have constant coefficient one, cancel common factors, and require all their zeros to lie outside the closed unit disk. With positive innovation variance these conditions give a causal, invertible, minimal Gaussian process representation. To prove identifiability, compare two such models with the same time-series spectral density. The ratio of their scaled rational transfer functions and its reciprocal are analytic on a neighbourhood of the closed unit disk and have modulus one on its boundary. The maximum modulus principle makes the ratio constant; its positive value at zero forces equality of the innovation variances and equality of the rational functions. Coprimality and the constant-coefficient normalization then force equality of both polynomials. Without the root restrictions, reciprocal root reflection of an ARMA representation and a rescaling of the innovation variance can preserve the spectrum, so the unrestricted parameters are not identifiable.
Canonical pseudometric of a Gaussian process Created 2026-10-06 Updated 2026-11-05
The canonical distance is the L2 norm of an increment of a Gaussian process. It is a pseudometric, since distinct parameters may represent equal random variables almost surely. Its metric covering number appears in the Dudley entropy integral. For standard fractional Brownian motion, ; a normalization with twice the covariance function multiplies this distance by .
Carl Edward Rasmussen Created 2026-10-13 Updated 2026-11-05
Coauthor with Christopher K. I. Williams of a textbook on Gaussian processes for machine learning. It treats regression through probability distributions over functions, linking covariance choices with predictive means and uncertainties.[ref]
Christopher K. I. Williams Created 2026-10-13 Updated 2026-11-05
Coauthor with Carl Edward Rasmussen of a textbook on Gaussian processes for machine learning, developing covariance-based regression and the corresponding predictive uncertainty.[ref]
A centered continuous stationary scalar Gaussian process of positive variance can be Markov only if its covariance satisfies the displayed equation for . Gaussian conditioning makes the future conditional mean ; the Markov property then forces the equation by conditioning a future observation on the present and past. Conversely this equation makes the Gaussian residual future independent of the past, proving the Markov property. In the nondegenerate decaying case the covariance is a single exponential. A sum with positive coefficients and distinct positive decay rates fails the equation and hence is not scalar Markov.
Covariance function Created 2026-09-28 Updated 2026-10-14
For a real-valued stochastic process with finite second moments, write . Its covariance function is
The diagonal gives the variance at each time; off-diagonal values record linear dependence between times. For the independent standard normal random variables above, , whereas the shared Gaussian process has for all . For a fair simple random walk on the integer line, ; Brownian motion has the continuous-time counterpart . These formulas quantify how much random history the observations share.
Every covariance function is a positive-semidefinite kernel. Indeed, for any real coefficients ,
Thus a proposed covariance function must make every finite covariance matrix positive semidefinite. For a Gaussian process, the mean function and the covariance function determine every finite-dimensional distribution, including degenerate ones such as the shared process. Conversely, any real positive-semidefinite kernel satisfying , together with a specified mean function, defines consistent multivariate normal distributions; the Kolmogorov extension theorem then constructs a Gaussian process. For a general stochastic process, covariance alone does not determine the full law. The exceptional completeness of this description is what makes Gaussian process regression possible with a mean and a covariance model.
On a real Hilbert space , this centered jointly Gaussian process is linear in its test vector and has the displayed covariance, with independent increments in time. Equivalently choose an orthonormal basis and independent real Brownian motions , and define stochastic integration by whenever . In infinite dimension it need not be an -valued random variable; the integral is the meaningful object.
A Cylindrical Wiener process on a real separable Hilbert space is a linear family of jointly Gaussian processes , indexed by , with continuous trajectories and
Linearity can be understood in the space of square-integrable random variables. For a unit vector , is standard Brownian motion; orthogonal directions give independent Brownian motions. The definition specifies all finite collections of observations without requiring a -valued random position.[ref]
For an orthonormal basis , let be independent standard Brownian motions. Then
converges in mean square for each , since its variance is . The formal vector is different. Its first coordinates have expected squared norm ; for fixed , the strong law of large numbers gives almost surely. The squared norm therefore diverges in infinitely many dimensions. Coordinate observations exist even though the proposed random vector does not belong to .
A Hilbert-Schmidt operator repairs this mismatch: converges as an -valued square-integrable random variable when is finite.[ref] For example, on , setting gives expected squared norm . This smoothing makes an actual random position, while retaining infinitely many noisy coordinates. It is useful when defining stochastic partial differential equations and their noise terms.
If a continuous local martingale starts at zero and has deterministic continuous quadratic variation , the Dambis-Dubins-Schwarz theorem gives . It is therefore a centered Gaussian process. The converse follows from independent increments of a Gaussian martingale.
For a centered Gaussian process with covariance function , twice continuous differentiability of and differentiability of at zero imply a differentiable modification of a stochastic process on compact intervals. The even function satisfies , so the mean-square derivative has variance of increments . A normal distribution has fourth absolute moment three times the square of its variance, so Kolmogorov continuity theorem gives a continuous modification of . The mean-square fundamental theorem of calculus identifies its path integral with a modification of a stochastic process of . This argument does not require integrability of or continuity of .
Dudley entropy integral Created 2026-10-03 Updated 2026-11-05
Let be a centered Gaussian process with a separable version: its relevant suprema are captured by a countable dense index subset. Give the canonical pseudometric of a Gaussian process , and assume it is totally bounded. With metric covering number and diameter , the Dudley entropy integral bounds
for a universal constant . Finiteness of the integral gives a finite expected supremum. For an absolute supremum, include an anchor: . The anchor term cannot generally be omitted.
The proof builds successively finer nets. Write each increment as a telescoping sum between neighboring net scales, bound each finite Gaussian maximum by its scale times the square root of the logarithmic net size, and sum over scales. This becomes the displayed entropy integral.[ref]
The version assumption is substantive. If is uniform on and , every finite vector is zero almost surely and is therefore a degenerate Gaussian vector; the canonical pseudometric is identically zero. Yet for every outcome. Its all-zero separable modification has supremum zero. Finite-dimensional laws alone do not control the supremum of an arbitrary uncountable version.
Exponential covariance function Created 2026-09-24 Updated 2026-11-05
The exponential covariance function on the real line is
Here is the marginal variance and the correlation time. For , the centered stationary Gaussian process with this covariance function is the Ornstein-Uhlenbeck process solving
with the initial value independent of future Brownian motion increments. The explicit stochastic-integral solution gives . Its conditional future equals the exponentially decayed present plus independent Gaussian noise, giving the Markov property.
The cusp at zero determines its roughness. For ,
which diverges. Thus a continuous version exists, but it has no mean-square derivative of a Gaussian process. A smooth-looking covariance away from the diagonal does not imply differentiable paths.[ref]
F-Brownian bridge 2026-10-14
For a cumulative distribution function and a standard Brownian bridge , the F-Brownian bridge is the centred Gaussian process . Its covariance is , by substituting into the bridge covariance. It can jump when jumps. Its law is tight in the supremum norm, since composition with is a continuous map from the separable space of continuous bridge paths.