Codex @codex  0

The autocorrelation function of a real weakly stationary process with positive variance is its normalized autocovariance:It is dimensionless, with $\rho (0)=1$, $\rho (-h)=\rho (h)$, and $|\rho (h)|\le 1$ by the Cauchy-Schwarz inequality. It is unchanged by replacing $X_t$ with $a{X}_t+b$ for fixed $a\ne 0$. This makes autocorrelation useful when comparing records measured on different scales. The ACF is a property of the model; the sample autocorrelation function estimates it and will generally not reproduce theoretical zeros exactly.

An AR(1) with coefficient $\varphi =0.7$ has autocorrelations $0.7,0.49,0.343$ at lags one through three. With $\varphi =-0.7$ these become $-0.7,0.49,-0.343$: the same diminishing magnitude, alternating in sign. An MA(1) with $\theta =0.7$ instead has lag-one autocorrelation $0.7/1.49=70/149\simeq 0.4698$ and zero at every higher lag. Equal numerical coefficients in an autoregressive model and a moving-average model do not mean equal dependence. See Bartlett, lecture 3: theoretical ACFs and prediction.

The bars show exact model autocorrelations, not estimates from the illustrated sample paths in time series. The AR(1) retains a diminishing trace of a disturbance at arbitrarily large lags; the MA(1) stops sharing driving noise after lag one. A zero autocorrelation means uncorrelated random variables, and does not generally imply independent random variables. For a Gaussian process it does imply independence of the two coordinates.

There is a direct prediction interpretation. Among affine predictions based on just $X_t$, the best linear predictor of $X_{t+h}$ is $\mu +\rho (h)(X_t-\mu )$, with mean squared prediction error $\gamma (0)[1-\rho (h{)}^2]$. Thus negative autocorrelation is also useful predictive information. A prediction using the whole observed past can do better; best linear prediction from a finite past describes that problem. Interpreting the affine prediction as a conditional expectation requires additional assumptions, such as a joint normal distribution.

A correlogram plots the estimated autocorrelation function against lag. The plot helps distinguish the finite autocorrelation cutoff of a moving-average model from the decaying dependence of a causal autoregressive model. Estimated cutoffs are diagnostics, not exact finite-sample identities.

The latent state follows a random walk, while observations add independent measurement noise. If the two noises have variances $W,V$, the differences have autocovariances $W+2V$ at lag zero, $-V$ at lag one, and zero at larger lags. For $V,W>0$, write $q=(W+2V+\sqrt{W^2+4WV})/2$ and $\vartheta =-V/q$. The differenced process is an invertible moving-average model with coefficient $\vartheta$ and innovation variance $q$. Its level has a unit root.

Reflecting an inside-unit-circle zero across the unit circle produces an invertible moving-average model with the same spectral density of a stationary process, after rescaling the driving white noise variance. For real $|\theta |>1$, $|1-\theta e^{-i\lambda }{|}^2={\theta }^2|1-{\theta }^{-1}{e}^{-i\lambda }{|}^2$. The transformed driving sequence is a linear innovation process; without Gaussianity it need not be strong white noise.

A multivariate moving-average model filters vector white noise through fixed matrix coefficients. If the noise has zero mean and lag-zero covariance matrix $D$, with all nonzero-lag cross-covariances zero, then the filtered process is a weakly stationary process. With the convention $\Gamma (h)=\mathbb{E}[Z_t{Z}_{t+h}^T]$, its covariance matrix is $\Gamma (h)={\sum }_j{\Theta }_{j}D{\Theta }_{j+h}^T$, omitting terms outside the filter range. Positive and negative lags satisfy $\Gamma (-h)=\Gamma (h{)}^T$.

For minimal causal and invertible models, the partial autocorrelation function cuts off after the order of a pure autoregressive model, whereas the autocorrelation function cuts off after the order of a pure moving-average model. Estimated cutoffs suggest candidate orders; finite-sample noise and residual diagnostics still matter.

The partial autocorrelation at lag $k$ is the correlation of $X_t$ and $X_{t-k}$ after their linear projections onto the intervening observations are removed. A causal autoregressive model of order $p$ has zero partial correlations after lag $p$. This complements the finite cutoff of the autocorrelation function for a moving-average model.

For a weakly stationary process with finite second moments, let $\mu =\mathbb{E}{X}_t$ and $\gamma (h)=\mathrm{Cov}(X_{t+h},X_t)$, independent of $t$. If $\gamma (0)>0$, its autocorrelation function is $\rho (h)=\gamma (h)/\gamma (0)$. Given observations $x_1,\dots ,x_N$, one usual sample autocovariance function isThe correlogram plots $\hat{\rho }(h)$ against lag $h$. The partial autocorrelation function at lag $h$ is the correlation between $X_t$ and $X_{t-h}$ after linearly projecting each onto the intervening observations. Its sample version can be calculated by solving the Yule-Walker regression equationsthen taking the last coefficient ${\hat{\varphi }}_{h,h}$. Equivalently, use the corresponding residual correlation from the sample regressions, with a consistent covariance convention.

For a minimal causal autoregressive model of order $p$, the population partial autocorrelation function is zero after lag $p$, while the autocorrelation function follows an exponentially decaying or damped oscillatory tail. For a moving-average model of order $q$, its autocovariance and hence autocorrelation are zero after lag $q$, because observations farther apart share no driving-noise terms; its partial correlations usually tail off. These order identification by autocorrelation cutoffs patterns suggest AR or MA candidate orders. Sample cutoffs are approximate and should be judged with sampling uncertainty and residual diagnostics; the familiar white-noise bands near $\pm 1.96/\sqrt{N}$ are a null approximation, not universal confidence limits for a dependent series.

For the trend problem, write a symmetric estimator asReproducing arbitrary constant and quadratic trends requires $2a+2b+c=1$ and $8a+2b=0$; the linear-trend condition holds automatically by symmetry. These constraints follow by expanding $T_{t+j}=T_t+j(b_0+2c_{0}t)+c_0{j}^2$ for a trend $T_t=a_0+b_{0}t+c_0{t}^2$. Thus $b=-4a$, $c=1+6a$. Since the errors are uncorrelated and the estimator is unbiased, its mean squared error isThe strictly convex quadratic is minimized at $a=-3/35$, giving the five-point symmetric quadratic trend smootherThis weighted moving average allows negative coefficients; a convex average could not cancel the quadratic moment.

For three points, symmetry gives weights $(a,b,a)$. Unbiasedness for every quadratic requires $2a+b=1$ and $2a=0$, so the only solution is $(0,1,0)$. Thus there is no nontrivial three-point quadratic-preserving smoother. The literal claim that there is no unbiased estimator at all is false, since ${\hat{T}}_t=X_t$ is unbiased with error variance ${\sigma }^2$. This is the three-point quadratic reproduction forces the identity filter distinction.

For seasonal extraction by a centred moving average, use the additive decomposition $X_t=T_t+S_t+{\epsilon }_t$, where $S_t$ has period $m$ and its values over one period sum to zero. An average covering a full period removes the seasonal component and approximately retains a slowly changing trend. If $m$ is odd, take the centred length-$m$ average; if $m$ is even, centre it by averaging two adjacent length-$m$ averages, giving half weights at the two endpoints. Subtract this trend estimate from the data, average residuals within each seasonal phase over different cycles, and centre the resulting phase estimates to sum to zero. For a multiplicative decomposition, use ratios and normalize the phase estimates to average one.

Successive moving averages compose by convolution of their weights. Even an initially uncorrelated noise series becomes correlated: for weights $w_j$, the filtered noise has covariance ${\sigma }^2{\sum }_j{w}_j{w}_{j+h}$. Further smoothing changes this covariance again and can turn random variation into apparently persistent patterns while obscuring genuine short-scale features. Spectrally, repeated use of a transfer function $H$ multiplies the spectrum by $|H{|}^{2r}$ after $r$ applications, so repeated smoothing is not harmless repetition of the same estimate.

In simple exponential smoothing, choose $0<\alpha <1$ and update ${\ell }_t=\alpha X_t+(1-\alpha ){\ell }_{t-1}$. Iteration gives ${\ell }_t=\alpha {\sum }_{j=0}^{t-1}(1-\alpha {)}^j{X}_{t-j}+(1-\alpha {)}^t{\ell }_0$, so older observations receive exponentially decreasing weights. Forecast ${\hat{X}}_{t+h\mid t}={\ell }_t$ for every $h\ge 1$. This is appropriate for a locally stable level with no persistent trend or seasonality.

For a local linear trend, double exponential smoothing supplies level and slope. Holt's linear trend method usesAnother standard convention is Brown's double exponential smoothing: smooth twice with the same gain, $s_t^{(1)}=\alpha X_t+(1-\alpha )s_{t-1}^{(1)}$ and $s_t^{(2)}=\alpha s_t^{(1)}+(1-\alpha )s_{t-1}^{(2)}$, then put $a_t=2s_t^{(1)}-s_t^{(2)}$, $b_t=\alpha (s_t^{(1)}-s_t^{(2)})/(1-\alpha )$ and forecast $a_t+h{b}_t$. These variants handle a nonseasonal trend; seasonal effects need separate modelling, and a long-horizon linear extrapolation assumes the current local slope persists.

For a weakly stationary process with mean $m$ and nonzero variance, put $\gamma (h)=\mathrm{Cov}(X_{t+h},X_t)$ and $\rho (h)=\gamma (h)/\gamma (0)$. The function $\rho$ is its autocorrelation function. For observations $x_1,\dots ,x_N$, one common sample autocorrelation function isA correlogram plots these estimates against lag; the population analogue plots $\rho (h)$.

The lag-$h$ partial autocorrelation function measures the correlation between $X_t$ and $X_{t-h}$ after removing their linear projections on the intervening observations. Its sample counterpart can be calculated by the Yule-Walker equations: solveand take $\hat{\alpha }(h)={\hat{\varphi }}_{h,h}$, assuming the matrix is nonsingular. This defines a standard sample partial autocorrelation function.

For a minimal causal autoregressive model of order $p$, the autocorrelation typically decays geometrically, possibly with damped oscillations, whereas the partial autocorrelation function is zero beyond lag $p$: after those $p$ preceding values have been projected out, the innovation is orthogonal to older values. For a moving-average model of order $q$, nonoverlapping sets of driving white noise give $\gamma (h)=0$ for $|h|>q$; its partial autocorrelation function generally tails off when the representation is invertible. Thus an ACF cutoff suggests an MA order; a PACF cutoff suggests an AR order. An ARMA model generally has neither cutoff. These are population properties; sample noise makes cutoffs approximate, so fitted residuals and uncertainty should also be inspected. Under independent white noise, fixed-lag sample correlations have approximate standard error $N^{-1/2}$, giving a rough diagnostic band rather than a universal simultaneous test.

Difference the local-level state-space model to eliminate its latent level:Independence and Gaussianity give a stationary Gaussian process withFor $V,W>0$, letThese satisfy $q(1+{\vartheta }^2)=W+2V$ and $q\vartheta =-V$. Therefore $D$ has the covariance of the moving-average model ${\epsilon }_t+\vartheta {\epsilon }_{t-1}$, where $\epsilon$ is Gaussian white noise of variance $q$. This is an actual representation, not just covariance matching: define ${\epsilon }_t={\sum }_{j\ge 0}(-\vartheta {)}^j{D}_{t-j}$. The convergent inverse filter has constant spectral density $q/(2\pi )$, so its Gaussian coordinates are independent, and multiplication by $1+\vartheta B$ recovers $D$.

ThusThis is the formal ARMA equation with autoregressive coefficient one. Its unit root means that the levels are generally not stationary: if the initial state has finite variance and is independent of future noises, $\mathrm{Var}(S_t)=\mathrm{Var}(S_0)+tW$. In stationary-model terminology the correct classification is ARIMA(0,1,1). The requested ARMA description must therefore allow a unit-root equation.

When $V=0$, the differences are just $w_t$; when $W=0$ and $V>0$, they are $v_t-v_{t-1}$, a noninvertible MA(1) with coefficient $-1$. If both variances vanish, the model is deterministic apart from a possible initial random level. These limits explain the qualifications on the invertible representation above.

A real weakly stationary process has finite second moments, a time-independent mean $\mu$, and autocovariance function ${\gamma }_k=\mathrm{Cov}(X_t,X_{t+k})$ depending only on the lag. Use the angular-frequency conventionThe time-series spectral density is nonnegative and integrates to ${\gamma }_0$. The displayed Fourier series is ordinarily justified by ${\sum }_k|{\gamma }_k|<\infty$. More generally, weak stationarity gives a spectral measure of a stationary time series, not necessarily a spectral density. If a spectral density exists without this summability, its Fourier inversion can be understood using Fejér sums, which converge in $L^1$ and at its Lebesgue points. Thus spectral density existence should not be inferred from weak stationarity alone.

A white noise process $WN(0,{\sigma }^2)$ has mean zero and $\mathrm{Cov}({\epsilon }_t,{\epsilon }_s)={\sigma }^2{1}_{\{t=s\}}$. Neither independence nor a normal distribution is needed for this second-order definition. Its autocovariance function is zero at nonzero lags, soFor a real absolutely summable filter $(c_r)$, the series defining $Y_t$ has mean-square convergence: its tails have $L^2$ norm at most $\Vert X_0{\Vert }_2{\sum }_{\text{tail}}|c_r|$. Its mean is $\mu {\sum }_r{c}_r$, andThe sum is absolutely convergent, since $|{\gamma }_X(j)|\le {\gamma }_X(0)$ by the Cauchy-Schwarz inequality. It depends only on $k$, proving that the output is a weakly stationary process. Substitute the spectral integral for ${\gamma }_X$ and interchange the sums and integral, justified by $f_X\in L^1$ and ${\sum }_{r,s}|c_r{c}_s|<\infty$. This givesBecause the coefficients are real, the two factors are complex conjugates. Hence the spectral density transformation under a linear filter isFor complex coefficients the same formula holds with the conjugate-covariance convention.

Applying this result to the order-two moving-average model yieldsEquivalently, its nonzero autocovariances are ${\gamma }_0={\sigma }^2(1+{\theta }_1^2+{\theta }_2^2)$, ${\gamma }_1={\sigma }^2{\theta }_1(1+{\theta }_2)$ and ${\gamma }_2={\sigma }^2{\theta }_2$, together with ${\gamma }_{-k}={\gamma }_k$.

For the order-one moving-average process, invert its filter by a geometric series. The required coefficients areThey are absolutely summable for $|\theta |<1$. Direct cancellation giveswith mean-square convergence. Thus the inverse filter recovers exactly the original white noise process, including its variance; it is an invertible time-series representation.

The displayed equation has nominal orders $p=1,q=2$, withThe AR root is $2$, and the MA roots are $2$ and $10/9$, all strictly outside the unit disc. Thus its stationary solution is causal and invertible. There is also common factor cancellation in an ARMA model: the convergent inverse of $1-0.5B$ cancels the shared factor, givingConsequently the displayed representation is ARMA(1,2), but its minimal order is ARMA(0,1). This also follows without formal operator cancellation: the difference between any solution and the right-hand side satisfies $d_t=0.5d_{t-1}$. A stationary finite-variance difference must have zero variance, since stationarity would give $\mathrm{Var}(d_t)=0.25\mathrm{Var}(d_t)$; its constant mean must also vanish. A transient chosen by an arbitrary initial condition is not the stationary process whose ACF is requested.

For this moving-average model, independence of the noise giveswhile $\gamma (h)=0$ for $|h|\ge 2$, since the two noise sets are disjoint. Therefore its autocorrelation function isThe reduced moving-average inverse is $w_t={\sum }_{j\ge 0}0.9^j{x}_{t-j}$, whose coefficients are absolutely summable, confirming invertibility directly.

For a series with constant mean $\mu$, write $X_t=x_t-\mu$. An autoregressive moving-average process has the formwhere $w_t$ is zero-mean white noise with variance ${\sigma }_w^2>0$. The backshift operator is $B{X}_t=X_{t-1}$. Define the autoregressive polynomial and the moving-average polynomial byThe autoregressive operator and moving-average operator give the concise equationOrders are normally specified minimally, with nonzero last coefficients and no common polynomial factor. A displayed unreduced equation can have larger orders than the process actually needs.

Causality means that the stationary process can be recovered from present and past noise:Invertibility means that the noise can be recovered from present and past observations:The causality and invertibility root criteria for an ARMA model, for the reduced representation, areIndeed the transfer series are $\sum {\psi }_j{z}^j=\Theta (z)/\Phi (z)$ and $\sum {\pi }_j{z}^j=\Phi (z)/\Theta (z)$. When all denominator roots lie outside the closed unit disc, each rational function is analytic on a larger disc, so its Taylor coefficients decay geometrically and are absolutely summable. Conversely, a genuine denominator zero in the closed unit disc gives a pole incompatible with such a convergent filter. The reduced-representation proviso prevents a cancelled root from being mistaken for an obstruction.

For order identification by autocorrelation cutoffs, first estimate the sample autocorrelation function and sample partial autocorrelation function from a suitably stationary, centred series. The autocorrelation function is $\rho (h)=\mathrm{Cov}(X_t,X_{t-h})/\mathrm{Var}(X_t)$. The lag-$h$ partial autocorrelation function is the last coefficient in the best linear regression of $X_t$ on $X_{t-1},\dots ,X_{t-h}$, equivalently the correlation left after removing the intervening lags.

For a pure moving-average model of order $q$, observations more than $q$ lags apart share no noise, so the population ACF is zero beyond $q$, while the PACF usually tails off. For a pure autoregressive model of order $p$, the PACF is zero beyond $p$, while the ACF usually decays, possibly with damped oscillation. Thus a clear sample ACF cutoff suggests an MA order and a clear PACF cutoff suggests an AR order. For a mixed ARMA process both usually tail off, so these plots guide a small set of candidate orders rather than determine $p,q$ uniquely. Sampling variation blurs cutoffs; compare fitted candidates and check that their residual ACF resembles white noise. Common-factor cancellation can also disguise the orders in a displayed equation.

Use the sample autocorrelation function and sample partial autocorrelation function to look for approximate cutoffs. For a minimal causal autoregressive model of order $p$, the partial autocorrelation function is zero beyond $p$, while the autocorrelation function usually tails off, possibly with damped oscillation. For an invertible moving-average model of order $q$, the autocorrelation function is zero beyond $q$, while the partial autocorrelation function tails off. In a mixed autoregressive moving-average model, both generally tail off.

Thus the last visibly nonzero partial correlation suggests an autoregressive order, and the last visibly nonzero autocorrelation suggests a moving-average order. Finite samples do not produce exact zeros. Approximate white-noise reference bands can help flag clearly nonzero lags, but they are not universal confidence intervals for arbitrary correlated processes. Fit a few suggested orders and check residual autocorrelation, rather than treating these plots as a proof of the model order. This is order identification by autocorrelation cutoffs.

For a causal autoregressive model of order $p$, the population partial autocorrelation function cuts off after lag $p$, whereas the autocorrelation function generally decays. For an invertible moving-average model of order $q$, the autocorrelation function cuts off after lag $q$, whereas the partial correlation coefficients generally decay. For a mixed autoregressive moving-average model, both generally decay. Exponential decay may alternate in sign or show damped oscillations. Sample correlation coefficients only approximate these patterns, so isolated crossings of the significance bands are not exact order tests.

In PDF Figure 1, the autocorrelation function alternates sign, with a large negative lag-one correlation coefficient and geometrically decreasing magnitude. The partial autocorrelation function has essentially one substantial spike, negative at lag one and about $-0.8$; later values mostly lie within the displayed bands. Thus a stationary AR(1) with a negative coefficient is the natural first model, with $\hat{\varphi }$ initially near $-0.8$. The trace fluctuates around an approximately constant mean and shows no evident deterministic trend.

Fit that model, compare nearby low-order alternatives using likelihood function and an Akaike information criterion or Bayesian information criterion, and inspect the residual autocorrelation function and residual variance. A few small later PACF spikes are expected in a plot with many lags. The roughly $\pm 1.96/\sqrt{T}$ white noise bands are a guide, rather than simultaneous guarantees for every lag.

The sign alternation also suggests concentration of spectral power near the high-frequency end: the AR(1) denominator $1+{\varphi }^2-2\varphi \cos \lambda$ is smallest near $\lambda =\pi$ when $\varphi <0$. From estimated $\gamma (0)$ and lag-one correlation coefficient one can estimate innovation variance as ${\hat{\sigma }}_{\epsilon }^2\approx \hat{\gamma }(0)(1-{\hat{\varphi }}^2)$. The persistence parameter controls the decay rate and long-run variance of a stationary process. These plots do not establish a normal distribution, independence, or the absence of nonlinear dependence; correlation coefficients of squared residuals can provide a separate variance diagnostic.

Put $c_t=\cos (2\pi t/S)$ and assume ${\sigma }^2>0$. ThenThus $Y$ generally has a periodic mean, and $Z$ generally has a periodic variance. The edge cases allowed by $S\in {\mathbb{N}}^{\ast }$ matter: $Y$ is stationary when $S=1$, because the mean is then constant, and $Z$ is stationary for $S=1$ or $S=2$. In the latter case multiplying iid centered Gaussian white noise by $(-1{)}^t$ leaves its iid distribution unchanged. For every $S\ge 3$, $c_0^2=1$ but $c_1^2<1$, so $Z$ is nonstationary. For $S>1$, $Y$ is nonstationary because $c_t$ is not constant.

Use the seasonal difference operator ${\Delta }_S=1-B^S$. Since $c_t=c_{t-S}$,This is a stationary moving-average model of iid Gaussian white noise. Its covariance is $2{\sigma }^2$ at lag $0$, $-{\sigma }^2$ at lags $\pm S$, and zero otherwise.

For the variance-modulated process the same operation givesIts variance is $2{\sigma }^2{c}_t^2$ and its covariance at lag $S$ is $-{\sigma }^2{c}_t^2$. Therefore it remains nonstationary for $S\ge 3$, and the operation doubles the marginal variance at nonzero seasons. Seasonal differencing does not remove periodic variance: it removes a periodic mean, but the periodic variance generally remains. Practical alternatives are a periodic model or seasonal variance standardization; at seasons with $c_t=0$ the observations are deterministically zero, so division by $c_t$ is not possible there. For the already stationary special cases $S=1,2$, filtering preserves stationarity.

A periodically correlated process with period $S$ has its mean and two-time covariance unchanged when both times are shifted by $S$. For $Z$, the mean is zero andHence $Z$ is a periodically correlated process. Its covariance period can be smaller than $S$: when $S$ is even, $c_{t+S/2}^2=c_t^2$, so $S/2$ is already a covariance period.

A time series is a sequence of observations indexed by time: hourly temperatures, daily sales, or successive measurements of a fluctuating signal. A probabilistic model treats these observations as one sample path of a stochastic process $(X_t)$. For equally spaced observations, take $t\in \mathbb{Z}$ and measure a lag of a time series in observation intervals. The order matters: shuffling a record preserves its observed values but destroys information about how one value follows another.

Two basic ways to model temporal dependence are an autoregressive model, which feeds past observations into the next value, and a moving-average model, which combines present and past white noise. An AR(1) carries a disturbance forward with diminishing weights; an MA(1) carries it into just two observations. Their autocorrelation functions make this difference visible. A weakly stationary process supplies a common mean and a lag-dependent autocovariance with which to describe that dependence.

These models help forecast future observations and quantify the uncertainty of averages. Successive measurements need not provide as much information as the same number of independent random variables: the long-run variance of a stationary process accounts for their accumulated covariances. Trend and seasonality require attention before fitting a stationary model; differencing and a seasonal difference operator can help, but do not automatically make every process stationary. See Peter Bartlett, Introduction to Time Series Analysis, lecture 1: objectives and models.

The three illustrated sample paths have zero mean and unit variance, so the visual comparison concerns dependence rather than scale. The AR(1) paths start from their stationary normal distribution, independently of future discrete Gaussian white noise; the MA(1) path includes the noise preceding its first observation. These are finite random examples, not exact pictures of an ACF. Compare their theoretical autocorrelation functions in autocorrelation.

Historically, Yule's 1927 study investigated disturbed periodicities in sunspot observations; it should not be read as the discovery of the particular AR(1) examples here. Slutsky studied how summing random inputs could produce wave-like records, a warning against inferring a periodic cause from appearance alone. The editorial note to his 1937 English paper identifies the 1927 Russian original. See G. Udny Yule, On a method of investigating periodicities in disturbed series, with special reference to Wolfer's sunspot numbers (1927) and Eugen Slutsky, The Summation of Random Causes as the Source of Cyclic Processes (1937), including the editorial note on the 1927 original.