Stationary process

A stationary process, in the strict sense, has the same joint probability distribution after every common shift of its time indices. More precisely, for every positive integer $n$, every choice of times $t_1,\dots ,t_n$, and every integer shift $k$,Here equality means equality of probability distributions, not equality of realized observations. A sample path can fluctuate, stay positive for a while, or show accidental runs while its generating law remains stationary. Strong white noise is a basic example: shifting time preserves every joint law of its independent and identically distributed random variables.

The distinction between strict stationarity and weak stationarity is useful. The former preserves all finite joint laws; the latter preserves only the finite mean and covariance structure. Strict stationarity with finite second moments implies weak stationarity, but strict stationarity itself does not require finite variance. A Gaussian process with weak stationarity is also strictly stationary, because each of its finite-dimensional distributions, a joint normal distribution, is determined by its mean vector and covariance matrix.

Every finite-dimensional distribution of a strictly stationary process is invariant under a common shift of its time indices. If second moments are finite, strict stationarity implies a weakly stationary process. A constant mean and lag-dependent covariance alone do not imply strict stationarity.

Let ${\mathcal{F}}_{-\infty }^0$ and ${\mathcal{F}}_n^{\infty }$ be the sigma-algebras generated by the past and future observations. The mixing coefficient is $\alpha (n)={\sup }_{A\in {\mathcal{F}}_{-\infty }^0,B\in {\mathcal{F}}_n^{\infty }}|P(A\cap B)-P(A)P(B)|$. Strong mixing means $\alpha (n)\to 0$. Appropriate quantitative mixing and moment conditions can imply a central limit theorem; absolute summability of autocovariance alone does not.

A stationary path law is ergodic when every time-shift-invariant event has probability zero or one. The Birkhoff ergodic theorem then identifies integrable time averages with deterministic ensemble expectations. A shared random scale multiplying iid noise gives a stationary counterexample: it is uncorrelated across distinct times, but the path retains information about the random scale.

A process is nonstationary if some finite-dimensional distribution changes under a common time shift. With finite second moments, failure of a constant mean or variance, or failure of lag-only covariance, already rules out a weakly stationary process. A periodic mean or periodic variance can instead give a periodically correlated process.

A real stochastic process $(X_t{)}_{t\in \mathbb{Z}}$ is a weakly stationary process if it has finite second moments, a constant expected value $\mu$, and an autocovariance depending only on the lag of a time series:In particular its variance is the same at every time, $\mathrm{Var}(X_t)=\gamma (0)$. This is often called second-order stationarity. It makes observations from different parts of a record comparable through the same autocorrelation function. Bartlett uses this second-order meaning of stationarity in Introduction to Time Series Analysis, lecture 2: mean, covariance, and stationarity.

For a worked contrast, let ${\epsilon }_t$ be centered white noise of variance ${\sigma }^2>0$. The process $X_t=5+{\epsilon }_t$ has mean $5$, variance ${\sigma }^2$, and zero autocovariance at nonzero lags, so it is weakly stationary. The process $Y_t=t+{\epsilon }_t$ has exactly the same covariance structure, but its mean is $t$, so it is not weakly stationary. Constant variance alone is insufficient.

Weak stationarity need not imply strict stationarity. For example, choose independent random variables at all integer times, using a standard normal distribution at even times and a Rademacher distribution at odd times. Every coordinate has zero mean and unit variance, and distinct coordinates have zero covariance. Nevertheless a shift by one changes the marginal probability distribution from continuous to two-point. This also illustrates why weak white noise need not be identically distributed.

An absolutely summable sequence of coefficients defines $Y_t={\sum }_s{a}_s{X}_{t-s}$ in mean square. It remains weakly stationary and has covariance ${\gamma }_k^Y={\sum }_{s,u}{a}_s{a}_u{\gamma }_{k-s+u}$. Its frequency response is $\alpha (\omega )={\sum }_s{a}_s{e}^{is\omega }$.

For absolutely summable linear-filter coefficients, the generating function is the displayed Laurent series, absolutely convergent on the unit circle. If the filter is one-sided, it is a power series analytic inside the unit disk. A two-sided filter need not have a nonempty larger annulus of convergence. Substituting the backshift operator gives the formal notation $Y=A(B)X$.

The transfer function is the filter generating function evaluated on the unit circle, with a sign convention matching the Fourier representation. Filtering a sinusoid of angular frequency $\lambda$ multiplies its complex amplitude by $H_A(\lambda )$. Its modulus is the filter gain; filtering a weakly stationary process multiplies its time-series spectral density by $|H_A(\lambda ){|}^2$. Using the opposite Fourier sign conjugates the response for real coefficients and leaves that squared modulus unchanged.

Two absolutely summable linear filters of a stationary time series compose by convolution: $c_r={\sum }_j{b}_j{a}_{r-j}$ and ${\sum }_r|c_r|\le ({\sum }_r|a_r|)({\sum }_j|b_j|)$. Absolute convergence in the space of square-integrable random variables justifies exchanging the sums. The frequency responses multiply, giving the spectral density transformation under a linear filter twice.

For an absolutely summable linear filter of a stationary time series, the filter gain is the modulus of its frequency response $a(\lambda )={\sum }_r{a}_r{e}^{ir\lambda }$. It multiplies a sinusoidal amplitude, while its square multiplies the spectral density of a stationary process. A zero removes the corresponding frequency; the filter gain of a composition is the product of the two filter gains.

Filtering multiplies the time-series spectral density by the squared modulus of the frequency response. Insert the spectral integral into the covariance double sum and interchange sums using absolute summability of the filter. This holds even when the input covariances are not absolutely summable, provided the spectral measure has a density.

The symmetric filter with weights $(-1,2,4,2,-1)/6$ has unit gain at zero frequency, but its largest gain is $7/6$ at signed frequencies $\pm \pi /3$. Applied $k$ times, it multiplies a spectral density by $|H{|}^{2k}$. Thus repetition amplifies and selects a period-six oscillation instead of merely suppressing short-scale noise. On a two-sided frequency domain both peaks must be retained.

The spectral measure of a weakly stationary process is the finite nonnegative measure whose Fourier coefficients are its autocovariances. Its total mass is the process variance. A time-series spectral density exists exactly when this measure is absolutely continuous with respect to Lebesgue measure.

For uncorrelated zero-mean unit-variance amplitudes $A,B$, the process $A\cos ({\omega }_{0}t)+B\sin ({\omega }_{0}t)$ has covariance $\cos ({\omega }_{0}k)$ and hence the displayed two-atom spectral measure of a stationary time series. Independence or Gaussian amplitudes are unnecessary for weak stationarity. A phase shift need not preserve the whole joint amplitude distribution, so strict stationarity cannot be inferred from these second moments alone.

The cumulative function of the finite positive spectral measure of a stationary time series is nondecreasing and right-continuous, with total mass equal to the variance. For an absolutely continuous measure, $F(\lambda )={\int }_{-\pi }^{\lambda }f(u)du$, where $f$ is the time-series spectral density. Under angular-frequency normalization, ${\gamma }_k={\int }_{-\pi }^{\pi }{e}^{ik\lambda }dF(\lambda )$ and, when the covariance series is absolutely summable, $f(\lambda )=(2\pi {)}^{-1}{\sum }_{k\in \mathbb{Z}}{\gamma }_k{e}^{-ik\lambda }$. Atoms describe persistent periodic components and cannot be represented by an ordinary density.

A second-order stationary complex random field on the line has uncentered correlation $C(\zeta )=\mathbb{E}[E(z+\zeta )\overline{E(z)}]$. When this correlation is continuous, its Fourier transform is a finite positive measure $F$ characterized by $C(\zeta )=\int e^{i\nu \zeta }F(d\nu )$. Its total mass is $\mathbb{E}|E(z){|}^2$. If it has a density, $S(\nu )=(2\pi {)}^{-1}\int C(\zeta )e^{-i\nu \zeta }d\zeta$. A constant mean $m$ contributes $|m{|}^2{\delta }_0$; the remaining measure is the covariance spectrum. This uncentered convention is useful for coherent and diffuse wave fields.

For the Fourier transform ${\hat{E}}_L(\nu )=(2\pi {)}^{-1}{\int }_{-L/2}^{L/2}E(z)e^{-i\nu z}dz$ of a stationary random field, the periodogram measure $(2\pi /L)\mathbb{E}|{\hat{E}}_L(\nu ){|}^{2}d\nu$ converges weakly to the spectral measure of a stationary random field. The expected squared modulus is $(4{\pi }^2{)}^{-1}{\int }_{-L}^L(L-|\zeta |)C(\zeta )e^{-i\nu \zeta }d\zeta$. The factor $2\pi /L$ follows from Parseval identity, and preserves the total power per unit length. It also avoids meaningless squares of Dirac delta distributions for coherent plane-wave components.

With absolutely summable autocovariance, the asymptotic variance of the sample mean satisfiesThe sum is signed, rather than a sum of absolute values. It can be zero: the first difference of strong white noise has telescoping partial sums. A central limit theorem needs further dependence assumptions.

When the long-run variance of a stationary process is positive, matching the variance of its average to an average of independent observations gives $T_{\mathrm{eff}}\simeq T\gamma (0)/{\sum }_h\gamma (h)$. Positive aggregate correlation reduces this size; negative aggregate correlation can increase it beyond the observation count. This extends the variance interpretation of effective sample size of a Markov chain.

When its autocovariance is absolutely summable, a weakly stationary process has spectral density $f(\lambda )=(2\pi {)}^{-1}{\sum }_h\gamma (h)e^{-ih\lambda }$. The inverse relation is $\gamma (h)={\int }_{-\pi }^{\pi }{e}^{ih\lambda }f(\lambda )d\lambda$. A linear filter multiplies this density by the squared modulus of its frequency response.

Under angular-frequency normalization, the displayed time-series spectral density gives ${\gamma }_0={\pi }^2$ and ${\gamma }_k=2(1-(-1{)}^k)/k^2$ for nonzero integer $k$. This follows from the even integral $2{\int }_0^{\pi }(\pi -\lambda )\cos (k\lambda )d\lambda$. All nonzero even-lag covariances vanish while the odd-lag covariances are positive.

For zero-mean real weakly stationary processes whose cross-covariances are summable, the cross-spectrum is the Fourier series of their lagged covariance, with the displayed convention. Reality gives $f_{XY}(-\lambda )=\overline{f_{XY}(\lambda )}$, but the cross-spectrum need not be real or even. Exchanging the two series gives $f_{YX}(\lambda )=\overline{f_{XY}(\lambda )}$. The entries form a Hermitian spectral density of a stationary process matrix.

For real stationary processes, the one-sided density on $[0,\pi ]$ is twice the usual two-sided density. Its integral is the variance and ${\gamma }_k={\int }_0^{\pi }f(\omega )\cos (k\omega )d\omega$. When covariances are absolutely summable, $f(\omega )={\pi }^{-1}({\gamma }_0+2{\sum }_{k\ge 1}{\gamma }_k\cos (k\omega ))$.

For a real weakly stationary process, restricting its even angular-frequency density $f$ to $[0,\pi ]$ gives $\gamma (k)=2{\int }_0^{\pi }f(\omega )\cos (k\omega )d\omega$. Folding both frequency halves into one density gives $g=2f$ and removes that factor two. Consequently white noise of variance $v$ has restricted density $v/(2\pi )$ and folded density $v/\pi$.

A weakly stationary process has a time-series spectral density precisely when its spectral measure of a stationary time series is absolutely continuous with respect to Lebesgue measure. Absolute summability of its autocovariances is sufficient and yields a continuous Fourier-series density. It is not necessary for existence; for a general integrable density, Fejér sums give an $L^1$ recovery formula.

In cycles per observation, frequency lies in $[-1/2,1/2]$ and $\gamma (h)=\int e^{2\pi ih\omega }{f}_c(\omega )d\omega$. Compared with angular-frequency density $f_a$, one has $f_c(\omega )=2\pi f_a(2\pi \omega )$. In particular, white noise of variance ${\sigma }^2$ has $f_c={\sigma }^2$.

A centered weakly stationary process has a representation $X_t={\int }_{-1/2}^{1/2}{e}^{2\pi it\omega }dZ(\omega )$ with orthogonal random increments. Their variance measure is the spectral measure. When it has density $f$, the autocovariance is $\gamma (h)=\int e^{2\pi ih\omega }f(\omega )d\omega$.

For a centered record of length $T$, the periodogram is the squared modulus of its discrete Fourier transform, with normalizationUnder suitable short-memory assumptions its expectation approaches the spectral density of a stationary process, but its variance generally does not vanish. Smoothing nearby frequencies can provide an estimate with statistical consistency.

At an interior frequency, the real and imaginary Fourier coefficients of a short-memory Gaussian time series converge jointly to independent centered normal variables, each with variance $s(\omega )/2$, where $s$ is the one-sided spectral density. Their squared sum therefore converges to $s(\omega ){\chi }_2^2/2$. Its variance tends to $s(\omega {)}^2$, so an unsmoothed periodogram is not consistent when $s(\omega )>0$. Averaging nearby ordinates can reduce this persistent variance.

For a centered stationary real Gaussian sequence, let $A_T$ and $B_T$ be its cosine and sine coefficients with normalization $1/\sqrt{\pi T}$ at a nonzero Fourier frequency. Grouping the double covariance sum by lag and extending each lag sum to a whole record leaves at most $2k$ omitted bounded products. The full cosine-sine sum is zero, giving the bound. If $\sum k|{\gamma }_k|<\infty$, the two coefficients become asymptotically uncorrelated, and their limiting joint normal distribution consequently has independent components.

Average neighboring periodogram ordinates with nonnegative weights summing to one. Let the number $L_T$ of averaged ordinates tend to infinity while their frequency span, of order $L_T/T$, tends to zero. For a smooth spectral density and suitable short-memory or fourth-order dependence conditions, the bias tends to zero and variance is of order $1/L_T$. Smooth second-order spectral density alone does not ensure this: a time-constant random scale multiplying independent white noise has a smooth unconditional spectrum but a nonergodic random limiting local power.

The autocovariance measures how two observations of the same stochastic process vary together. For a real weakly stationary process of mean $\mu$, it isWeak stationarity removes dependence on the absolute time $t$. Without it, the covariance function generally needs two time arguments. Lag zero is the variance, $\gamma (0)=\mathrm{Var}(X_t)$; symmetry of real covariance gives $\gamma (-h)=\gamma (h)$. Positive values favor deviations from the mean in the same direction; negative values favor opposite directions. The units are the square of the observation units. See Bartlett, lecture 2: autocovariance examples.

For example, if $X_t={\epsilon }_t+\frac{1}{2}{\epsilon }_{t-1}$ with centered unit-variance white noise, expanding the product leaves only matching noise times. Thus $\gamma (0)=1+1/4=5/4$, $\gamma (1)=1/2$, and $\gamma (h)=0$ for $|h|\ge 2$. This MA(1) has a positive lag-one autocovariance because adjacent observations share one disturbance. Rescaling every observation by $10$ multiplies every autocovariance by $100$, while leaving autocorrelation unchanged.

The autocovariance is useful beyond comparing pairs: it determines the variance of any finite linear combination of observations. For the sample mean ${\overline{X}}_n$,Each separation $h$ occurs in $n-h$ pairs. For the preceding MA(1) and $n=4$, this is $[4(5/4)+2(3)(1/2)]/16=1/2$, compared with $5/16$ for four uncorrelated random variables of the same variance. This is the finite-record counterpart of the long-run variance of a stationary process.

For a known zero mean, ${\hat{\gamma }}_n(h)=n^{-1}{\sum }_{t=1}^{n-h}{Y}_t{Y}_{t+h}$ for $0\le h<n$. Weakly stationary process assumptions give expectation $(1-h/n)\gamma (h)$. Dividing by $n-h$ instead gives an unbiased estimate at each individual lag, although the resulting collection need not form a positive semidefinite covariance sequence.

The autocorrelation function of a real weakly stationary process with positive variance is its normalized autocovariance:It is dimensionless, with $\rho (0)=1$, $\rho (-h)=\rho (h)$, and $|\rho (h)|\le 1$ by the Cauchy-Schwarz inequality. It is unchanged by replacing $X_t$ with $a{X}_t+b$ for fixed $a\ne 0$. This makes autocorrelation useful when comparing records measured on different scales. The ACF is a property of the model; the sample autocorrelation function estimates it and will generally not reproduce theoretical zeros exactly.

An AR(1) with coefficient $\varphi =0.7$ has autocorrelations $0.7,0.49,0.343$ at lags one through three. With $\varphi =-0.7$ these become $-0.7,0.49,-0.343$: the same diminishing magnitude, alternating in sign. An MA(1) with $\theta =0.7$ instead has lag-one autocorrelation $0.7/1.49=70/149\simeq 0.4698$ and zero at every higher lag. Equal numerical coefficients in an autoregressive model and a moving-average model do not mean equal dependence. See Bartlett, lecture 3: theoretical ACFs and prediction.

The bars show exact model autocorrelations, not estimates from the illustrated sample paths in time series. The AR(1) retains a diminishing trace of a disturbance at arbitrarily large lags; the MA(1) stops sharing driving noise after lag one. A zero autocorrelation means uncorrelated random variables, and does not generally imply independent random variables. For a Gaussian process it does imply independence of the two coordinates.

There is a direct prediction interpretation. Among affine predictions based on just $X_t$, the best linear predictor of $X_{t+h}$ is $\mu +\rho (h)(X_t-\mu )$, with mean squared prediction error $\gamma (0)[1-\rho (h{)}^2]$. Thus negative autocorrelation is also useful predictive information. A prediction using the whole observed past can do better; best linear prediction from a finite past describes that problem. Interpreting the affine prediction as a conditional expectation requires additional assumptions, such as a joint normal distribution.

A correlation time measures how rapidly a stationary stochastic process loses temporal dependence. Specify whether it means an exponential decay time, an integral of the normalized autocorrelation, or another width. For $R(t)=R(0)e^{-|t|/\tau }$, the exponential time and the one-sided integral ${\int }_0^{\infty }R(t)dt/R(0)$ both equal $\tau$.

The partial autocorrelation at lag $k$ is the correlation of $X_t$ and $X_{t-k}$ after their linear projections onto the intervening observations are removed. A causal autoregressive model of order $p$ has zero partial correlations after lag $p$. This complements the finite cutoff of the autocorrelation function for a moving-average model.

Replace the autocovariance in the finite-lag linear prediction equations by the sample autocovariance function. The last regression coefficient at order $k$ estimates the lag-$k$ partial autocorrelation function.

The sample autocorrelation function replaces the mean and lagged covariance in $\rho (h)$ by their empirical counterparts.

A correlogram plots the estimated autocorrelation function against lag. The plot helps distinguish the finite autocorrelation cutoff of a moving-average model from the decaying dependence of a causal autoregressive model. Estimated cutoffs are diagnostics, not exact finite-sample identities.

For the AR(1) and MA(1) formulas here, white noise means a real sequence indexed by all integer times withThese are the weak white noise assumptions: zero mean, a common finite positive variance, and uncorrelated random variables at distinct times. Independence and a normal distribution are additional assumptions. Strong white noise requires independent and identically distributed random variables; discrete Gaussian white noise is the jointly Gaussian special case. Some definitions allow a constant nonzero mean; subtract it before using the centered formulas here.

For a concrete dependent example, let $A$ equal $1$ or $2$ with equal probabilities, independently of a sequence of independent Rademacher random variables $Z_t$, and put ${\epsilon }_t=A{Z}_t$. Its mean is zero, its variance is $\mathbb{E}[A^2]=5/2$, and its nonzero-lag autocovariances vanish. Nevertheless every $|{\epsilon }_t|$ equals the same $A$: seeing one magnitude determines all the others. This is white noise with dependence invisible to its ACF, and is also an example of the issue discussed in ergodic stationary process.

Discrete Gaussian white noise is a jointly Gaussian sequence with expected value zero and covariance ${\sigma }^2$ at lag zero and zero at other lags. Joint normality makes the components independent. This is a sequence of ordinary random variables, unlike the generalized continuous-time white-noise process.

Continuous-time Gaussian white noise is a random distribution rather than an ordinary finite-variance function at every time. Integrating it against deterministic kernels gives Gaussian random variables with covariance $q\int f(s)g(s)ds$. For independent isotropic components, a three-dimensional dot covariance has amplitude $3q$. An ordinary rapidly decaying force covariance approximates this model only above its correlation time.

A sequence of independent and identically distributed random variables with zero mean and finite variance is strong white noise. It is weak white noise, but need not have a normal distribution.

A white noise sequence has zero mean, a common finite variance and zero autocovariance at every nonzero lag. This requires uncorrelated random variables, rather than independent random variables.