Time series

A time series is a sequence of observations indexed by time: hourly temperatures, daily sales, or successive measurements of a fluctuating signal. A probabilistic model treats these observations as one sample path of a stochastic process $(X_t)$. For equally spaced observations, take $t\in \mathbb{Z}$ and measure a lag of a time series in observation intervals. The order matters: shuffling a record preserves its observed values but destroys information about how one value follows another.

Two basic ways to model temporal dependence are an autoregressive model, which feeds past observations into the next value, and a moving-average model, which combines present and past white noise. An AR(1) carries a disturbance forward with diminishing weights; an MA(1) carries it into just two observations. Their autocorrelation functions make this difference visible. A weakly stationary process supplies a common mean and a lag-dependent autocovariance with which to describe that dependence.

These models help forecast future observations and quantify the uncertainty of averages. Successive measurements need not provide as much information as the same number of independent random variables: the long-run variance of a stationary process accounts for their accumulated covariances. Trend and seasonality require attention before fitting a stationary model; differencing and a seasonal difference operator can help, but do not automatically make every process stationary. See Peter Bartlett, Introduction to Time Series Analysis, lecture 1: objectives and models.

The three illustrated sample paths have zero mean and unit variance, so the visual comparison concerns dependence rather than scale. The AR(1) paths start from their stationary normal distribution, independently of future discrete Gaussian white noise; the MA(1) path includes the noise preceding its first observation. These are finite random examples, not exact pictures of an ACF. Compare their theoretical autocorrelation functions in autocorrelation.

Historically, Yule's 1927 study investigated disturbed periodicities in sunspot observations; it should not be read as the discovery of the particular AR(1) examples here. Slutsky studied how summing random inputs could produce wave-like records, a warning against inferring a periodic cause from appearance alone. The editorial note to his 1937 English paper identifies the 1927 Russian original. See G. Udny Yule, On a method of investigating periodicities in disturbed series, with special reference to Wolfer's sunspot numbers (1927) and Eugen Slutsky, The Summation of Random Causes as the Source of Cyclic Processes (1937), including the editorial note on the 1927 original.

A lag of a time series is a separation between observation indices. In an equally spaced time series, lag $h=3$ compares observations three intervals apart: for hourly readings this is three hours, while for daily readings it is three days. Positive and negative lags describe the same separation in opposite directions. The autocovariance and autocorrelation of a real weakly stationary process are even functions of lag.

A symmetric three-point smoother with weights $(a,b,a)$ reproduces constants only if $2a+b=1$. Reproducing quadratic trends also requires $2a=0$, so $a=0$ and $b=1$. Thus the identity estimator $X_t$ is unbiased, but there is no nontrivial three-point linear smoother reproducing every quadratic. An assertion excluding even the identity estimator is false. This clarifies why the five-point symmetric quadratic trend smoother is the first symmetric nontrivial example.

For observations $X_t=T_t+{\epsilon }_t$ with a quadratic trend and uncorrelated equal-variance errors, a symmetric five-point linear filter of a stationary time series has weights $(a,b,c,b,a)$. Reproducing every quadratic requires $2a+2b+c=1$ and $8a+2b=0$. Substitution gives $b=-4a$, $c=1+6a$ and error variance ${\sigma }^2(70a^2+12a+1)$. Its unique minimum occurs at $a=-3/35$, producing the displayed weights and variance $17{\sigma }^2/35$. Negative weights are allowed: this is a linear weighted smoother, not a convex average. Applying a linear filter to a deterministic trend plus stationary errors does not assert that the complete observation process is stationary.

For independent identically distributed continuous observations, count strict local maxima and minima at interior positions. Each turning indicator has probability $2/3$, adjacent indicators have joint probability $5/12$, indicators two positions apart have joint probability $9/20$, and more distant indicators are independent. Summing these covariances gives the displayed variance for $T\ge 4$. The indicators form a bounded 2-dependent sequence, giving an asymptotically normal standardized count. Too few turns suggests persistent trends; too many suggests alternation. Ties and dependent observations require a different calibration.

With a fixed gain $0<K<1$, exponential smoothing recursively combines the newest observation with the preceding estimate. Iteration assigns geometric weights $K(1-K{)}^j$ to older observations, together with a diminishing weight on the initial estimate. The steady-state local-level Kalman gain gives a probabilistic interpretation of this recursion.

Double exponential smoothing supplements a smoothed level with a smoothed slope so that forecasts can extrapolate a local linear trend. Two common versions are Holt's linear trend method, with separate level and slope gains, and Brown's double exponential smoothing, which applies the same smoothing gain twice. They are different algorithms, not two names for one recursion. Neither version by itself models seasonal variation.

For $0<\alpha <1$, set $s_t^{(1)}=\alpha X_t+(1-\alpha )s_{t-1}^{(1)}$ and $s_t^{(2)}=\alpha s_t^{(1)}+(1-\alpha )s_{t-1}^{(2)}$. Use the displayed corrected level and slope, then forecast $a_t+h{b}_t$. For an exact linear trend after initialization transients, the first and second smooths have lags $(1-\alpha )/\alpha$ and twice that lag; their difference recovers the slope and the corrected level removes the lag. This is distinct from Holt's linear trend method.

Update the level and slope by ${\ell }_t=\alpha X_t+(1-\alpha )({\ell }_{t-1}+b_{t-1})$ and $b_t=\beta ({\ell }_t-{\ell }_{t-1})+(1-\beta )b_{t-1}$, with gains in $(0,1)$. The first update compares the new observation with the previous level extrapolated one step; the second smooths the latest level increment. The displayed forecast extrapolates that local slope. This is a double exponential smoothing method for a nonseasonal trending time series.

For $0<\alpha <1$, update a local level by the displayed recursion and forecast every future horizon by ${\ell }_t$. Iteration gives geometrically decreasing weights $\alpha (1-\alpha {)}^j$ on past observations and a vanishing contribution from the initial level. It is useful for a locally stable level without a persistent trend or seasonality. A trend causes lag, motivating double exponential smoothing.

A second-order stationary time series decomposes into a deterministic component and a one-sided square-summable linear filter of a stationary time series of its mutually orthogonal linear innovations. Normalize $c_0=1$. For a causal invertible ARMA process with no deterministic component, the driving white noise is this innovation process: observations and innovations span the same closed past subspace, while the next innovation is orthogonal to it.

The best mean-square linear predictor is orthogonal projection onto the span of the available centered observations, plus the known mean. Orthogonal innovations give a convenient basis of that span. Prediction at a future horizon must use only actually observed variables; it need not be a one-step predictor.

For a causal autoregressive process with mean $m$ and innovations orthogonal to the observed past, define ${\hat{X}}_{T,h}=X_{T+h}$ for $h\le 0$ and replace future innovations by zero. The displayed recursion is its best linear prediction from a finite past when at least the last $p$ observations are available. Future innovation sums are orthogonal to those observations, which proves the projection property. Mere white noise does not guarantee that a linear forecast equals a conditional mean: that stronger interpretation requires martingale-difference or independent innovations.

If the increments of $W$ have mean $\mu$ and centered increments satisfy a causal autoregressive process of order one with $|\varphi |<1$, sum their recursive forecasts to obtain the displayed expression. For fixed forecast origin, ${\hat{W}}_{T,k}/k\to \mu$. One must observe the last increment, requiring $T\ge 2$ or a known initial level $W_0$; a single observed level without an initial-level model does not identify that increment.

Starting observations at a finite time, subtract the best linear predictor based on the available earlier observations. The resulting residual is orthogonal to their linear span. The unit-triangular relation between observations and residuals makes the residuals an orthogonal basis of the same finite observation space. These differ from infinite-past innovations until the initialization effect disappears.

In a centered MA(1) process, $X_{T+2}$ is uncorrelated with every observation through time $T$. Its best linear predictor from that information is therefore zero, with error variance ${\sigma }^2(1+{\theta }^2)$. One-step innovation formulas using $X_{T+1}$ cannot be substituted when that observation is unavailable.

Starting at time one, the orthogonal residuals satisfy $U_t=X_t-{\lambda }_t{U}_{t-1}$, with ${\lambda }_t={\gamma }_1/{\phi }_{t-1}$ and ${\phi }_t={\gamma }_0-{\gamma }_1{\lambda }_t$. Earlier innovation directions have zero covariance with the new observation. Explicitly, with $S_m={\sum }_{j=0}^m{\theta }^{2j}$, ${\phi }_t={\sigma }^2{S}_t/S_{t-1}$ and ${\lambda }_t=\theta S_{t-2}/S_{t-1}$ for $t\ge 2$. These formulas follow by induction from $S_t=(1+{\theta }^2)S_{t-1}-{\theta }^2{S}_{t-2}$, with $S_0=1$ and $S_1=1+{\theta }^2$. This includes $\theta =\pm 1$, where ${\phi }_t={\sigma }^2(t+1)/t$.

For an MA(1) process, the recursion ${\lambda }_t=\theta /(1+{\theta }^2-\theta {\lambda }_{t-1})$ has candidate limits $\theta$ and $1/\theta$. The limit in $[-1,1]$ is $\theta$ when $|\theta |\le 1$ and $1/\theta$ otherwise. The limiting innovation variance is ${\sigma }^2\max (1,{\theta }^2)$, expressing the same covariance law through an invertible reciprocal representation.

The best mean-square linear predictor from a semi-infinite past is the orthogonal projection onto the closed linear span of past observations in the random-variable Hilbert space. For a causal invertible ARMA representation, past observations and past innovations generate the same closed span. The next linear innovation is orthogonal to it and is therefore the prediction error.

A state-space model separates a latent transition equation $S_t=F{S}_{t-1}+Z_t$ from an observation equation $Y_t=H{S}_t+W_t$. A complete specification includes the initial-state law and its relation to the noise sequences, not just the two equations. Linear second-order models specify covariance structure; Gaussian versions additionally prescribe joint Gaussian laws.

For an autoregressive moving-average model of order $(1,1)$, this state has observation row $F=(\varphi ,1,\theta )$ and transition matrixThe state noise is $(0,{\epsilon }_t,0{)}^{\top }$. Its first transition row reconstructs $X_{t-1}$ and its third row shifts the previous white noise value.

For $|\varphi |<1$ and centered independent white noise values, the state above has expected value zero and this stationary covariance matrix, where ${\gamma }_0={\sigma }^2(1+{\theta }^2+2\varphi \theta )/(1-{\varphi }^2)$. The cross-covariance with the future white noise value is zero and that with the contemporaneous white noise value is ${\sigma }^2$. Direct multiplication verifies $P=GP{G}^{\top }+Q$.

The latent state follows a random walk, while observations add independent measurement noise. If the two noises have variances $W,V$, the differences have autocovariances $W+2V$ at lag zero, $-V$ at lag one, and zero at larger lags. For $V,W>0$, write $q=(W+2V+\sqrt{W^2+4WV})/2$ and $\vartheta =-V/q$. The differenced process is an invertible moving-average model with coefficient $\vartheta$ and innovation variance $q$. Its level has a unit root.

In a linear Gaussian state-space model, let the predicted state mean and covariance be $m_t^{-},P_t^{-}$. The observation innovation process has conditional mean zero and covariance ${\Sigma }_t=A{P}_t^{-}{A}^{\top }+R$, with innovation ${ϵ}_t=y_t-A{m}_t^{-}$. The conditional multivariate normal distribution gives the predictive observation density. Multiplying these conditional densities by the chain rule gives the displayed likelihood function; equivalently the innovations are independent Gaussian vectors. The Kalman filter supplies $m_t^{-},P_t^{-}$ recursively, so one evaluates the exact observed-data likelihood function without integrating all latent states jointly.

For $|\varphi |<1$, initialize the state by $S_0={\sum }_{j\ge 0}{\varphi }^j{Z}_{-j}$, with variance ${\sigma }_z^2/(1-{\varphi }^2)$. It is orthogonal to future state noise and to observation noise orthogonal to all state noise. Under a Gaussian specification, take the initial state Gaussian with this variance and independent of future noises. Starting at zero instead gives a transient model.

An anticausal noise representation uses future rather than present and past innovations. For a two-sided AR(1) equation with $|\varphi |>1$, the stationary solution is $X_t=-{\sum }_{j\ge 1}{\varphi }^{-j}{Z}_{t+j}$. Thus stationary existence alone does not require causality.

A linear process is an L2-convergent white-noise filter $X_t=\mu +{\sum }_{j\in \mathbb{Z}}{\psi }_j{\epsilon }_{t-j}$, with square-summable coefficients. It is weakly stationary, and its autocovariance is ${\sigma }_{\epsilon }^2{\sum }_j{\psi }_j{\psi }_{j+h}$. A causal representation restricts the coefficients to $j\ge 0$; it is then an infinite moving-average representation.

When the transfer function is $(1+\theta z)/((1-rz)(1-sz))$ with distinct $r,s$ of modulus less than one, partial fractions give coefficients ${\psi }_j=a{r}^j+d{s}^j$, where $a+d=1$ and $-as-dr=\theta$. White-noise orthogonality gives covariance at lag $h\ge 0$ as ${\sigma }^2[a^2{r}^h/(1-r^2)+d^2{s}^h/(1-s^2)+ad(r^h+s^h)/(1-rs)]$. This combines stable recursion with a closed geometric-sum covariance.

An ARCH model writes $X_t={\sigma }_t{\epsilon }_t$ with standardized independent driving noise and a conditional variance ${\sigma }_t^2={\alpha }_0+{\sum }_{j=1}^p{\alpha }_j{X}_{t-j}^2$. Typically ${\alpha }_0>0$ and ${\alpha }_j\ge 0$ ensure a positive conditional variance. It allows a changing conditional scale even when $X$ is a stationary process.

With symmetric driving noise whose sign is independent of its magnitude, changing the sign of one driving variable changes the corresponding $X_t$ but leaves future conditional scales unchanged. Consequently $\mathrm{Cov}(X_t,f(X_{t+h}))=0$ for $h>0$ and square-integrable $f(X_{t+h})$. This conclusion uses symmetry, beyond the martingale difference sequence property.

For ${\alpha }_0>0$ and $0<{\alpha }_2<1$, the model $X_t=\sqrt{{\alpha }_0+{\alpha }_2{X}_{t-2}^2}{\epsilon }_t$ splits into independent even-time and odd-time chains in its stationary causal solution. With standard normal noise, $\mathbb{E}{X}_t^2={\alpha }_0/(1-{\alpha }_2)$, and its fourth moment is finite exactly when $3{\alpha }_2^2<1$.

The stationary squared process has the positive series $X_t^2={\alpha }_0{\sum }_{k\ge 0}{\alpha }_2^k{\prod }_{j=0}^k{\epsilon }_{t-2j}^2$. Even and odd observations therefore use disjoint families of independent random variables. This explains zero lag-one covariance of the squares despite dependence at lag two.

Volatility clustering means that large absolute observations tend to be followed by further large absolute observations, and small ones by small ones. In an ARCH process, persistence is in the conditional variance rather than necessarily in the signed observations.

Seasonality is systematic repetition at a calendar period. It can appear in a deterministic mean, in a periodic variance, or in dependence across seasons. A seasonal difference operator removes a fixed periodic mean but seasonal differencing does not remove periodic variance. A periodic-looking path alone does not prove a nonstationary process.

For an additive time series $X_t=T_t+S_t+{\epsilon }_t$ with period-$m$ seasonal component and ${\sum }_{j=1}^m{S}_j=0$, a centred average spanning one complete period removes the seasonal component while estimating the slowly varying trend. When $m$ is even, average two adjacent length-$m$ averages to centre the filter on an observation; the endpoints receive half the usual weights. Estimate each seasonal phase by averaging the corresponding detrended residuals across cycles, then subtract the mean of those phase averages to enforce the zero-sum convention. Multiplicative seasonality instead uses ratios and a mean-one convention. Repeated averaging is filter composition and induces serial dependence even from white noise.

A weakly stationary process can have very slowly decaying autocovariance, for example $\gamma (h)\sim C{h}^{-\beta }$ with $C>0$ and $0<\beta <1$. Its sample mean can require a normalization different from $\sqrt{T}$. Long memory is dependence over long lags, rather than a changing marginal distribution.

In a second-order sense a weakly stationary process has short memory when its autocovariance is absolutely summable. Then its spectral density of a stationary process is continuous and its sample-mean variance has the usual finite long-run variance of a stationary process. A central limit theorem still needs additional conditions.

A process has period-$S$ second-order statistics when $E{X}_{t+S}=E{X}_t$ and $\mathrm{Cov}(X_{t+S},X_{s+S})=\mathrm{Cov}(X_t,X_s)$ for all times. Its variance can vary by season, so it need not be a weakly stationary process. A periodic covariance period need not be the fundamental period.

The difference $\Delta X_t=X_t-X_{t-1}$ can remove a unit-root stochastic trend. A seasonal difference operator instead subtracts an observation a full seasonal period earlier. Neither operation automatically removes a changing variance.

For an integer period $S$, ${\Delta }_S{X}_t=X_t-X_{t-S}$ uses the backshift operator as $1-B^S$. It annihilates a deterministic period-$S$ mean. If $X_t=m_t+{\epsilon }_t$ with $m_{t+S}=m_t$ and strong white noise, the result is the stationary moving average ${\epsilon }_t-{\epsilon }_{t-S}$.

If $X_t=c_t{\epsilon }_t$ with a deterministic periodic scale $c_t=c_{t-S}$ and strong white noise of variance ${\sigma }^2$, then ${\Delta }_S{X}_t=c_t({\epsilon }_t-{\epsilon }_{t-S})$. Its variance is $2{\sigma }^2{c}_t^2$, still seasonal when $c_t^2$ varies. A periodic scale model or variance standardization is more appropriate than blindly applying differencing.

An innovation measures the part of the current observation not predicted from past observations. The linear innovation process uses orthogonal projection onto the past linear span. A conditional expectation predictor can use nonlinear information; the two predictors agree for a Gaussian process.

Observe $Y_t=\lambda t+W_t$, with an independent Gaussian random vector $\lambda$ of mean $m_0$ and covariance matrix $V_0$. Its posterior is Gaussian with mean $m_t=(I+t{V}_0{)}^{-1}(m_0+V_0{Y}_t)$ and covariance matrix $V_t=V_0(I+t{V}_0{)}^{-1}$. These formulas also hold for a singular positive semidefinite matrix $V_0$. When $V_0$ is invertible, completing the square in the Bayes' theorem gives $V_t^{-1}=V_0^{-1}+tI$ and $m_t=V_t(V_0^{-1}{m}_0+Y_t)$. The observed innovation process ${\hat{W}}_t=Y_t-{\int }_0^t{m}_{s}ds$ is a Brownian motion in the observation filtration, and $d{m}_t=V_{t}d{\hat{W}}_t$.

For the Gaussian Brownian drift filter with positive definite $V_0$, define $Z_t=\det (I+t{V}_0{)}^{1/2}\exp [-m_t^T{V}_t^{-1}{m}_t/2+m_0^T{V}_0^{-1}{m}_0/2]$. Because $V_t^{\prime }=-V_t^2$ and $d{m}_t=V_{t}d{\hat{W}}_t$, the Itô formula gives $d\log Z_t=-m_t^{T}d{\hat{W}}_t-|m_t{|}^{2}dt/2$. Thus $Z$ is a positive stochastic exponential, hence a nonnegative local martingale and a supermartingale. For a singular prior, the inverse-matrix expression is undefined; the same process can instead be defined by $Z_t=({\mathbb{E}}_{\lambda \sim N(m_0,V_0)}\exp [{\lambda }^T{Y}_t-t|\lambda {|}^2/2]{)}^{-1}$, with $Y_t$ treated as fixed inside this Gaussian integral.

For the Gaussian Brownian drift filter $Y_t=\alpha t+W_t$, with independent prior $\alpha \sim N(m_0,{\tau }_0^{-1})$, the observation law has densityrelative to driftless Wiener measure. The law having fixed observation drift $k$ has density $e^{k{Y}_t-k^{2}t/2}$ relative to the same reference. Their ratio is therefore a positive mean-one density on every finite horizon. In the observation filtration it satisfies $d{D}_t=-D_t(m_t-k)d{\hat{W}}_t$, where $m_t$ is the posterior mean and $\hat{W}$ the innovation process. This density-ratio argument establishes a true martingale without an unsupported global Novikov condition for the unbounded posterior mean.

For $Y_s=\lambda s+W_s$ and fixed $t>0$, subtracting $(s/t)Y_t$ leaves the Brownian bridge $W_s-(s/t)W_t$. It is independent of $(\lambda ,Y_t)$, since it is independent of $\lambda$ and has zero covariance with $W_t$ in their jointly Gaussian law. The observed path is determined by this bridge and its endpoint, so the bridge gives no further information about $\lambda$. Hence conditioning on the full observation filtration at $t$ gives the same posterior as conditioning on $Y_t$ alone.

For $X_t=W_t+\alpha t$ with an independent equiprobable drift $\alpha \in \{-a,a\}$, the likelihood ratio between the two drifts is $e^{2a{X}_t}$. Multiplying the normal distribution densities of successive increments proves this ratio for every observation partition; continuity then gives the full observed-path posterior. The Bayes' theorem gives $q_t=\mathbb{P}(\alpha =a\mid {\mathcal{F}}_t^X)=(1+e^{-2a{X}_t}{)}^{-1}$. Thus the observed drift is $a(2q_t-1)=a\tanh (a{X}_t)$. The process ${\hat{W}}_t=X_t-{\int }_0^{t}a\tanh (a{X}_s)ds$ is an observed-filtration martingale with quadratic variation $t$, so the Lévy characterization of Brownian motion makes it a Brownian motion. The observed state is a scaled diffusion with hyperbolic tangent drift.

For a square-integrable time series, let ${\mathcal{H}}_{t-1}$ be the closed linear span of its past. The innovation ${\epsilon }_t=X_t-{\mathrm{proj}}_{{\mathcal{H}}_{t-1}}{X}_t$ is orthogonal to every past linear observation. A causal invertible autoregressive moving-average model is driven by these innovations, which are weak white noise for a weakly stationary process.

A naive forecast repeats the latest observation at every future horizon. For a stationary mean-$\mu$ process its horizon-$h$ mean squared prediction error is $2[\gamma (0)-\gamma (h)]$. A forecast of the known mean has risk $\gamma (0)$, so persistence is not automatically better when lagged dependence is weak.

A causal time series can be expressed using present and past innovations, without future innovations. The autoregressive process of order one with $|\alpha |<1$ has the mean-square convergent causal representation $Y_t={\sum }_{j\ge 0}{\alpha }^j{\epsilon }_{t-j}$.

A causal time-series representation expresses a time series using present and past driving white noise. Its coefficients vanish at negative lags. Square summability ensures mean-square convergence for a white-noise input; absolute summability is the stronger stable-filter convention. Causality refers to the particular driving sequence, not merely to stationary existence.

A stationary process, in the strict sense, has the same joint probability distribution after every common shift of its time indices. More precisely, for every positive integer $n$, every choice of times $t_1,\dots ,t_n$, and every integer shift $k$,Here equality means equality of probability distributions, not equality of realized observations. A sample path can fluctuate, stay positive for a while, or show accidental runs while its generating law remains stationary. Strong white noise is a basic example: shifting time preserves every joint law of its independent and identically distributed random variables.

The distinction between strict stationarity and weak stationarity is useful. The former preserves all finite joint laws; the latter preserves only the finite mean and covariance structure. Strict stationarity with finite second moments implies weak stationarity, but strict stationarity itself does not require finite variance. A Gaussian process with weak stationarity is also strictly stationary, because each of its finite-dimensional distributions, a joint normal distribution, is determined by its mean vector and covariance matrix.

Every finite-dimensional distribution of a strictly stationary process is invariant under a common shift of its time indices. If second moments are finite, strict stationarity implies a weakly stationary process. A constant mean and lag-dependent covariance alone do not imply strict stationarity.

Let ${\mathcal{F}}_{-\infty }^0$ and ${\mathcal{F}}_n^{\infty }$ be the sigma-algebras generated by the past and future observations. The mixing coefficient is $\alpha (n)={\sup }_{A\in {\mathcal{F}}_{-\infty }^0,B\in {\mathcal{F}}_n^{\infty }}|P(A\cap B)-P(A)P(B)|$. Strong mixing means $\alpha (n)\to 0$. Appropriate quantitative mixing and moment conditions can imply a central limit theorem; absolute summability of autocovariance alone does not.

A stationary path law is ergodic when every time-shift-invariant event has probability zero or one. The Birkhoff ergodic theorem then identifies integrable time averages with deterministic ensemble expectations. A shared random scale multiplying iid noise gives a stationary counterexample: it is uncorrelated across distinct times, but the path retains information about the random scale.

A process is nonstationary if some finite-dimensional distribution changes under a common time shift. With finite second moments, failure of a constant mean or variance, or failure of lag-only covariance, already rules out a weakly stationary process. A periodic mean or periodic variance can instead give a periodically correlated process.

A real stochastic process $(X_t{)}_{t\in \mathbb{Z}}$ is a weakly stationary process if it has finite second moments, a constant expected value $\mu$, and an autocovariance depending only on the lag of a time series:In particular its variance is the same at every time, $\mathrm{Var}(X_t)=\gamma (0)$. This is often called second-order stationarity. It makes observations from different parts of a record comparable through the same autocorrelation function. Bartlett uses this second-order meaning of stationarity in Introduction to Time Series Analysis, lecture 2: mean, covariance, and stationarity.

For a worked contrast, let ${\epsilon }_t$ be centered white noise of variance ${\sigma }^2>0$. The process $X_t=5+{\epsilon }_t$ has mean $5$, variance ${\sigma }^2$, and zero autocovariance at nonzero lags, so it is weakly stationary. The process $Y_t=t+{\epsilon }_t$ has exactly the same covariance structure, but its mean is $t$, so it is not weakly stationary. Constant variance alone is insufficient.

Weak stationarity need not imply strict stationarity. For example, choose independent random variables at all integer times, using a standard normal distribution at even times and a Rademacher distribution at odd times. Every coordinate has zero mean and unit variance, and distinct coordinates have zero covariance. Nevertheless a shift by one changes the marginal probability distribution from continuous to two-point. This also illustrates why weak white noise need not be identically distributed.

An absolutely summable sequence of coefficients defines $Y_t={\sum }_s{a}_s{X}_{t-s}$ in mean square. It remains weakly stationary and has covariance ${\gamma }_k^Y={\sum }_{s,u}{a}_s{a}_u{\gamma }_{k-s+u}$. Its frequency response is $\alpha (\omega )={\sum }_s{a}_s{e}^{is\omega }$.

For absolutely summable linear-filter coefficients, the generating function is the displayed Laurent series, absolutely convergent on the unit circle. If the filter is one-sided, it is a power series analytic inside the unit disk. A two-sided filter need not have a nonempty larger annulus of convergence. Substituting the backshift operator gives the formal notation $Y=A(B)X$.

The transfer function is the filter generating function evaluated on the unit circle, with a sign convention matching the Fourier representation. Filtering a sinusoid of angular frequency $\lambda$ multiplies its complex amplitude by $H_A(\lambda )$. Its modulus is the filter gain; filtering a weakly stationary process multiplies its time-series spectral density by $|H_A(\lambda ){|}^2$. Using the opposite Fourier sign conjugates the response for real coefficients and leaves that squared modulus unchanged.

Two absolutely summable linear filters of a stationary time series compose by convolution: $c_r={\sum }_j{b}_j{a}_{r-j}$ and ${\sum }_r|c_r|\le ({\sum }_r|a_r|)({\sum }_j|b_j|)$. Absolute convergence in the space of square-integrable random variables justifies exchanging the sums. The frequency responses multiply, giving the spectral density transformation under a linear filter twice.

For an absolutely summable linear filter of a stationary time series, the filter gain is the modulus of its frequency response $a(\lambda )={\sum }_r{a}_r{e}^{ir\lambda }$. It multiplies a sinusoidal amplitude, while its square multiplies the spectral density of a stationary process. A zero removes the corresponding frequency; the filter gain of a composition is the product of the two filter gains.

Filtering multiplies the time-series spectral density by the squared modulus of the frequency response. Insert the spectral integral into the covariance double sum and interchange sums using absolute summability of the filter. This holds even when the input covariances are not absolutely summable, provided the spectral measure has a density.

The symmetric filter with weights $(-1,2,4,2,-1)/6$ has unit gain at zero frequency, but its largest gain is $7/6$ at signed frequencies $\pm \pi /3$. Applied $k$ times, it multiplies a spectral density by $|H{|}^{2k}$. Thus repetition amplifies and selects a period-six oscillation instead of merely suppressing short-scale noise. On a two-sided frequency domain both peaks must be retained.

The spectral measure of a weakly stationary process is the finite nonnegative measure whose Fourier coefficients are its autocovariances. Its total mass is the process variance. A time-series spectral density exists exactly when this measure is absolutely continuous with respect to Lebesgue measure.

For uncorrelated zero-mean unit-variance amplitudes $A,B$, the process $A\cos ({\omega }_{0}t)+B\sin ({\omega }_{0}t)$ has covariance $\cos ({\omega }_{0}k)$ and hence the displayed two-atom spectral measure of a stationary time series. Independence or Gaussian amplitudes are unnecessary for weak stationarity. A phase shift need not preserve the whole joint amplitude distribution, so strict stationarity cannot be inferred from these second moments alone.