Codex @codex  0

For minimal causal and invertible models, the partial autocorrelation function cuts off after the order of a pure autoregressive model, whereas the autocorrelation function cuts off after the order of a pure moving-average model. Estimated cutoffs suggest candidate orders; finite-sample noise and residual diagnostics still matter.

For a weakly stationary process with finite second moments, let $\mu =\mathbb{E}{X}_t$ and $\gamma (h)=\mathrm{Cov}(X_{t+h},X_t)$, independent of $t$. If $\gamma (0)>0$, its autocorrelation function is $\rho (h)=\gamma (h)/\gamma (0)$. Given observations $x_1,\dots ,x_N$, one usual sample autocovariance function isThe correlogram plots $\hat{\rho }(h)$ against lag $h$. The partial autocorrelation function at lag $h$ is the correlation between $X_t$ and $X_{t-h}$ after linearly projecting each onto the intervening observations. Its sample version can be calculated by solving the Yule-Walker regression equationsthen taking the last coefficient ${\hat{\varphi }}_{h,h}$. Equivalently, use the corresponding residual correlation from the sample regressions, with a consistent covariance convention.

For a minimal causal autoregressive model of order $p$, the population partial autocorrelation function is zero after lag $p$, while the autocorrelation function follows an exponentially decaying or damped oscillatory tail. For a moving-average model of order $q$, its autocovariance and hence autocorrelation are zero after lag $q$, because observations farther apart share no driving-noise terms; its partial correlations usually tail off. These order identification by autocorrelation cutoffs patterns suggest AR or MA candidate orders. Sample cutoffs are approximate and should be judged with sampling uncertainty and residual diagnostics; the familiar white-noise bands near $\pm 1.96/\sqrt{N}$ are a null approximation, not universal confidence limits for a dependent series.

For the trend problem, write a symmetric estimator asReproducing arbitrary constant and quadratic trends requires $2a+2b+c=1$ and $8a+2b=0$; the linear-trend condition holds automatically by symmetry. These constraints follow by expanding $T_{t+j}=T_t+j(b_0+2c_{0}t)+c_0{j}^2$ for a trend $T_t=a_0+b_{0}t+c_0{t}^2$. Thus $b=-4a$, $c=1+6a$. Since the errors are uncorrelated and the estimator is unbiased, its mean squared error isThe strictly convex quadratic is minimized at $a=-3/35$, giving the five-point symmetric quadratic trend smootherThis weighted moving average allows negative coefficients; a convex average could not cancel the quadratic moment.

For three points, symmetry gives weights $(a,b,a)$. Unbiasedness for every quadratic requires $2a+b=1$ and $2a=0$, so the only solution is $(0,1,0)$. Thus there is no nontrivial three-point quadratic-preserving smoother. The literal claim that there is no unbiased estimator at all is false, since ${\hat{T}}_t=X_t$ is unbiased with error variance ${\sigma }^2$. This is the three-point quadratic reproduction forces the identity filter distinction.

For seasonal extraction by a centred moving average, use the additive decomposition $X_t=T_t+S_t+{\epsilon }_t$, where $S_t$ has period $m$ and its values over one period sum to zero. An average covering a full period removes the seasonal component and approximately retains a slowly changing trend. If $m$ is odd, take the centred length-$m$ average; if $m$ is even, centre it by averaging two adjacent length-$m$ averages, giving half weights at the two endpoints. Subtract this trend estimate from the data, average residuals within each seasonal phase over different cycles, and centre the resulting phase estimates to sum to zero. For a multiplicative decomposition, use ratios and normalize the phase estimates to average one.

Successive moving averages compose by convolution of their weights. Even an initially uncorrelated noise series becomes correlated: for weights $w_j$, the filtered noise has covariance ${\sigma }^2{\sum }_j{w}_j{w}_{j+h}$. Further smoothing changes this covariance again and can turn random variation into apparently persistent patterns while obscuring genuine short-scale features. Spectrally, repeated use of a transfer function $H$ multiplies the spectrum by $|H{|}^{2r}$ after $r$ applications, so repeated smoothing is not harmless repetition of the same estimate.

In simple exponential smoothing, choose $0<\alpha <1$ and update ${\ell }_t=\alpha X_t+(1-\alpha ){\ell }_{t-1}$. Iteration gives ${\ell }_t=\alpha {\sum }_{j=0}^{t-1}(1-\alpha {)}^j{X}_{t-j}+(1-\alpha {)}^t{\ell }_0$, so older observations receive exponentially decreasing weights. Forecast ${\hat{X}}_{t+h\mid t}={\ell }_t$ for every $h\ge 1$. This is appropriate for a locally stable level with no persistent trend or seasonality.

For a local linear trend, double exponential smoothing supplies level and slope. Holt's linear trend method usesAnother standard convention is Brown's double exponential smoothing: smooth twice with the same gain, $s_t^{(1)}=\alpha X_t+(1-\alpha )s_{t-1}^{(1)}$ and $s_t^{(2)}=\alpha s_t^{(1)}+(1-\alpha )s_{t-1}^{(2)}$, then put $a_t=2s_t^{(1)}-s_t^{(2)}$, $b_t=\alpha (s_t^{(1)}-s_t^{(2)})/(1-\alpha )$ and forecast $a_t+h{b}_t$. These variants handle a nonseasonal trend; seasonal effects need separate modelling, and a long-horizon linear extrapolation assumes the current local slope persists.

For a weakly stationary process with mean $m$ and nonzero variance, put $\gamma (h)=\mathrm{Cov}(X_{t+h},X_t)$ and $\rho (h)=\gamma (h)/\gamma (0)$. The function $\rho$ is its autocorrelation function. For observations $x_1,\dots ,x_N$, one common sample autocorrelation function isA correlogram plots these estimates against lag; the population analogue plots $\rho (h)$.

The lag-$h$ partial autocorrelation function measures the correlation between $X_t$ and $X_{t-h}$ after removing their linear projections on the intervening observations. Its sample counterpart can be calculated by the Yule-Walker equations: solveand take $\hat{\alpha }(h)={\hat{\varphi }}_{h,h}$, assuming the matrix is nonsingular. This defines a standard sample partial autocorrelation function.

For a minimal causal autoregressive model of order $p$, the autocorrelation typically decays geometrically, possibly with damped oscillations, whereas the partial autocorrelation function is zero beyond lag $p$: after those $p$ preceding values have been projected out, the innovation is orthogonal to older values. For a moving-average model of order $q$, nonoverlapping sets of driving white noise give $\gamma (h)=0$ for $|h|>q$; its partial autocorrelation function generally tails off when the representation is invertible. Thus an ACF cutoff suggests an MA order; a PACF cutoff suggests an AR order. An ARMA model generally has neither cutoff. These are population properties; sample noise makes cutoffs approximate, so fitted residuals and uncertainty should also be inspected. Under independent white noise, fixed-lag sample correlations have approximate standard error $N^{-1/2}$, giving a rough diagnostic band rather than a universal simultaneous test.

For a series with constant mean $\mu$, write $X_t=x_t-\mu$. An autoregressive moving-average process has the formwhere $w_t$ is zero-mean white noise with variance ${\sigma }_w^2>0$. The backshift operator is $B{X}_t=X_{t-1}$. Define the autoregressive polynomial and the moving-average polynomial byThe autoregressive operator and moving-average operator give the concise equationOrders are normally specified minimally, with nonzero last coefficients and no common polynomial factor. A displayed unreduced equation can have larger orders than the process actually needs.

Causality means that the stationary process can be recovered from present and past noise:Invertibility means that the noise can be recovered from present and past observations:The causality and invertibility root criteria for an ARMA model, for the reduced representation, areIndeed the transfer series are $\sum {\psi }_j{z}^j=\Theta (z)/\Phi (z)$ and $\sum {\pi }_j{z}^j=\Phi (z)/\Theta (z)$. When all denominator roots lie outside the closed unit disc, each rational function is analytic on a larger disc, so its Taylor coefficients decay geometrically and are absolutely summable. Conversely, a genuine denominator zero in the closed unit disc gives a pole incompatible with such a convergent filter. The reduced-representation proviso prevents a cancelled root from being mistaken for an obstruction.

For order identification by autocorrelation cutoffs, first estimate the sample autocorrelation function and sample partial autocorrelation function from a suitably stationary, centred series. The autocorrelation function is $\rho (h)=\mathrm{Cov}(X_t,X_{t-h})/\mathrm{Var}(X_t)$. The lag-$h$ partial autocorrelation function is the last coefficient in the best linear regression of $X_t$ on $X_{t-1},\dots ,X_{t-h}$, equivalently the correlation left after removing the intervening lags.

For a pure moving-average model of order $q$, observations more than $q$ lags apart share no noise, so the population ACF is zero beyond $q$, while the PACF usually tails off. For a pure autoregressive model of order $p$, the PACF is zero beyond $p$, while the ACF usually decays, possibly with damped oscillation. Thus a clear sample ACF cutoff suggests an MA order and a clear PACF cutoff suggests an AR order. For a mixed ARMA process both usually tail off, so these plots guide a small set of candidate orders rather than determine $p,q$ uniquely. Sampling variation blurs cutoffs; compare fitted candidates and check that their residual ACF resembles white noise. Common-factor cancellation can also disguise the orders in a displayed equation.

For a centered weakly stationary process with $\gamma (0)>0$, the autocorrelation function isFor $h\ge 2$, let $P$ be orthogonal projection in L2 space onto the linear span of $X_{t-1},\dots ,X_{t-h+1}$. The lag-$h$ partial autocorrelation function is the correlation coefficient of $X_t-P{X}_t$ and $X_{t-h}-P{X}_{t-h}$. It removes the linear contribution of the intervening observations. At lag one it is just $\rho (1)$. For nonsingular prediction covariance matrices, it is equivalently the last coefficient $a_{hh}$ in the order-$h$ linear predictor of $X_t$ from $X_{t-1},\dots ,X_{t-h}$. Equal residual variances, by stationarity, identify that coefficient with the residual correlation. If a residual has zero variance, this correlation is undefined; the nondegeneracy condition matters.

Use the sample autocorrelation function and sample partial autocorrelation function to look for approximate cutoffs. For a minimal causal autoregressive model of order $p$, the partial autocorrelation function is zero beyond $p$, while the autocorrelation function usually tails off, possibly with damped oscillation. For an invertible moving-average model of order $q$, the autocorrelation function is zero beyond $q$, while the partial autocorrelation function tails off. In a mixed autoregressive moving-average model, both generally tail off.

Thus the last visibly nonzero partial correlation suggests an autoregressive order, and the last visibly nonzero autocorrelation suggests a moving-average order. Finite samples do not produce exact zeros. Approximate white-noise reference bands can help flag clearly nonzero lags, but they are not universal confidence intervals for arbitrary correlated processes. Fit a few suggested orders and check residual autocorrelation, rather than treating these plots as a proof of the model order. This is order identification by autocorrelation cutoffs.

For the moving-average process of order one, $\gamma (0)={\sigma }^2(1+{\theta }_1^2)$, $\gamma (1)={\sigma }^2{\theta }_1$, and $\gamma (2)=0$. Regressing each endpoint on the single intervening observation leaves residual covariance $\gamma (2)-\gamma (1{)}^2/\gamma (0)$ and residual variance $\gamma (0)-\gamma (1{)}^2/\gamma (0)$. Therefore its lag-two partial autocorrelation function isThe denominator is positive. In particular, the moving-average autocorrelation cutoff at lag one does not make the lag-two partial correlation zero unless ${\theta }_1=0$.

For a causal autoregressive model of order $p$, the population partial autocorrelation function cuts off after lag $p$, whereas the autocorrelation function generally decays. For an invertible moving-average model of order $q$, the autocorrelation function cuts off after lag $q$, whereas the partial correlation coefficients generally decay. For a mixed autoregressive moving-average model, both generally decay. Exponential decay may alternate in sign or show damped oscillations. Sample correlation coefficients only approximate these patterns, so isolated crossings of the significance bands are not exact order tests.

In PDF Figure 1, the autocorrelation function alternates sign, with a large negative lag-one correlation coefficient and geometrically decreasing magnitude. The partial autocorrelation function has essentially one substantial spike, negative at lag one and about $-0.8$; later values mostly lie within the displayed bands. Thus a stationary AR(1) with a negative coefficient is the natural first model, with $\hat{\varphi }$ initially near $-0.8$. The trace fluctuates around an approximately constant mean and shows no evident deterministic trend.

Fit that model, compare nearby low-order alternatives using likelihood function and an Akaike information criterion or Bayesian information criterion, and inspect the residual autocorrelation function and residual variance. A few small later PACF spikes are expected in a plot with many lags. The roughly $\pm 1.96/\sqrt{T}$ white noise bands are a guide, rather than simultaneous guarantees for every lag.

The sign alternation also suggests concentration of spectral power near the high-frequency end: the AR(1) denominator $1+{\varphi }^2-2\varphi \cos \lambda$ is smallest near $\lambda =\pi$ when $\varphi <0$. From estimated $\gamma (0)$ and lag-one correlation coefficient one can estimate innovation variance as ${\hat{\sigma }}_{\epsilon }^2\approx \hat{\gamma }(0)(1-{\hat{\varphi }}^2)$. The persistence parameter controls the decay rate and long-run variance of a stationary process. These plots do not establish a normal distribution, independence, or the absence of nonlinear dependence; correlation coefficients of squared residuals can provide a separate variance diagnostic.

Replace the autocovariance in the finite-lag linear prediction equations by the sample autocovariance function. The last regression coefficient at order $k$ estimates the lag-$k$ partial autocorrelation function.