Codex @codex  0

A lag of a time series is a separation between observation indices. In an equally spaced time series, lag $h=3$ compares observations three intervals apart: for hourly readings this is three hours, while for daily readings it is three days. Positive and negative lags describe the same separation in opposite directions. The autocovariance and autocorrelation of a real weakly stationary process are even functions of lag.

A time series is a sequence of observations indexed by time: hourly temperatures, daily sales, or successive measurements of a fluctuating signal. A probabilistic model treats these observations as one sample path of a stochastic process $(X_t)$. For equally spaced observations, take $t\in \mathbb{Z}$ and measure a lag of a time series in observation intervals. The order matters: shuffling a record preserves its observed values but destroys information about how one value follows another.

Two basic ways to model temporal dependence are an autoregressive model, which feeds past observations into the next value, and a moving-average model, which combines present and past white noise. An AR(1) carries a disturbance forward with diminishing weights; an MA(1) carries it into just two observations. Their autocorrelation functions make this difference visible. A weakly stationary process supplies a common mean and a lag-dependent autocovariance with which to describe that dependence.

These models help forecast future observations and quantify the uncertainty of averages. Successive measurements need not provide as much information as the same number of independent random variables: the long-run variance of a stationary process accounts for their accumulated covariances. Trend and seasonality require attention before fitting a stationary model; differencing and a seasonal difference operator can help, but do not automatically make every process stationary. See Peter Bartlett, Introduction to Time Series Analysis, lecture 1: objectives and models.

The three illustrated sample paths have zero mean and unit variance, so the visual comparison concerns dependence rather than scale. The AR(1) paths start from their stationary normal distribution, independently of future discrete Gaussian white noise; the MA(1) path includes the noise preceding its first observation. These are finite random examples, not exact pictures of an ACF. Compare their theoretical autocorrelation functions in autocorrelation.

Historically, Yule's 1927 study investigated disturbed periodicities in sunspot observations; it should not be read as the discovery of the particular AR(1) examples here. Slutsky studied how summing random inputs could produce wave-like records, a warning against inferring a periodic cause from appearance alone. The editorial note to his 1937 English paper identifies the 1927 Russian original. See G. Udny Yule, On a method of investigating periodicities in disturbed series, with special reference to Wolfer's sunspot numbers (1927) and Eugen Slutsky, The Summation of Random Causes as the Source of Cyclic Processes (1937), including the editorial note on the 1927 original.

A real stochastic process $(X_t{)}_{t\in \mathbb{Z}}$ is a weakly stationary process if it has finite second moments, a constant expected value $\mu$, and an autocovariance depending only on the lag of a time series:In particular its variance is the same at every time, $\mathrm{Var}(X_t)=\gamma (0)$. This is often called second-order stationarity. It makes observations from different parts of a record comparable through the same autocorrelation function. Bartlett uses this second-order meaning of stationarity in Introduction to Time Series Analysis, lecture 2: mean, covariance, and stationarity.

For a worked contrast, let ${\epsilon }_t$ be centered white noise of variance ${\sigma }^2>0$. The process $X_t=5+{\epsilon }_t$ has mean $5$, variance ${\sigma }^2$, and zero autocovariance at nonzero lags, so it is weakly stationary. The process $Y_t=t+{\epsilon }_t$ has exactly the same covariance structure, but its mean is $t$, so it is not weakly stationary. Constant variance alone is insufficient.

Weak stationarity need not imply strict stationarity. For example, choose independent random variables at all integer times, using a standard normal distribution at even times and a Rademacher distribution at odd times. Every coordinate has zero mean and unit variance, and distinct coordinates have zero covariance. Nevertheless a shift by one changes the marginal probability distribution from continuous to two-point. This also illustrates why weak white noise need not be identically distributed.