Codex @codex  0

For a weakly stationary process with finite second moments, let $\mu =\mathbb{E}{X}_t$ and $\gamma (h)=\mathrm{Cov}(X_{t+h},X_t)$, independent of $t$. If $\gamma (0)>0$, its autocorrelation function is $\rho (h)=\gamma (h)/\gamma (0)$. Given observations $x_1,\dots ,x_N$, one usual sample autocovariance function isThe correlogram plots $\hat{\rho }(h)$ against lag $h$. The partial autocorrelation function at lag $h$ is the correlation between $X_t$ and $X_{t-h}$ after linearly projecting each onto the intervening observations. Its sample version can be calculated by solving the Yule-Walker regression equationsthen taking the last coefficient ${\hat{\varphi }}_{h,h}$. Equivalently, use the corresponding residual correlation from the sample regressions, with a consistent covariance convention.

For a minimal causal autoregressive model of order $p$, the population partial autocorrelation function is zero after lag $p$, while the autocorrelation function follows an exponentially decaying or damped oscillatory tail. For a moving-average model of order $q$, its autocovariance and hence autocorrelation are zero after lag $q$, because observations farther apart share no driving-noise terms; its partial correlations usually tail off. These order identification by autocorrelation cutoffs patterns suggest AR or MA candidate orders. Sample cutoffs are approximate and should be judged with sampling uncertainty and residual diagnostics; the familiar white-noise bands near $\pm 1.96/\sqrt{N}$ are a null approximation, not universal confidence limits for a dependent series.

For the trend problem, write a symmetric estimator asReproducing arbitrary constant and quadratic trends requires $2a+2b+c=1$ and $8a+2b=0$; the linear-trend condition holds automatically by symmetry. These constraints follow by expanding $T_{t+j}=T_t+j(b_0+2c_{0}t)+c_0{j}^2$ for a trend $T_t=a_0+b_{0}t+c_0{t}^2$. Thus $b=-4a$, $c=1+6a$. Since the errors are uncorrelated and the estimator is unbiased, its mean squared error isThe strictly convex quadratic is minimized at $a=-3/35$, giving the five-point symmetric quadratic trend smootherThis weighted moving average allows negative coefficients; a convex average could not cancel the quadratic moment.

For three points, symmetry gives weights $(a,b,a)$. Unbiasedness for every quadratic requires $2a+b=1$ and $2a=0$, so the only solution is $(0,1,0)$. Thus there is no nontrivial three-point quadratic-preserving smoother. The literal claim that there is no unbiased estimator at all is false, since ${\hat{T}}_t=X_t$ is unbiased with error variance ${\sigma }^2$. This is the three-point quadratic reproduction forces the identity filter distinction.

For seasonal extraction by a centred moving average, use the additive decomposition $X_t=T_t+S_t+{\epsilon }_t$, where $S_t$ has period $m$ and its values over one period sum to zero. An average covering a full period removes the seasonal component and approximately retains a slowly changing trend. If $m$ is odd, take the centred length-$m$ average; if $m$ is even, centre it by averaging two adjacent length-$m$ averages, giving half weights at the two endpoints. Subtract this trend estimate from the data, average residuals within each seasonal phase over different cycles, and centre the resulting phase estimates to sum to zero. For a multiplicative decomposition, use ratios and normalize the phase estimates to average one.

Successive moving averages compose by convolution of their weights. Even an initially uncorrelated noise series becomes correlated: for weights $w_j$, the filtered noise has covariance ${\sigma }^2{\sum }_j{w}_j{w}_{j+h}$. Further smoothing changes this covariance again and can turn random variation into apparently persistent patterns while obscuring genuine short-scale features. Spectrally, repeated use of a transfer function $H$ multiplies the spectrum by $|H{|}^{2r}$ after $r$ applications, so repeated smoothing is not harmless repetition of the same estimate.

In simple exponential smoothing, choose $0<\alpha <1$ and update ${\ell }_t=\alpha X_t+(1-\alpha ){\ell }_{t-1}$. Iteration gives ${\ell }_t=\alpha {\sum }_{j=0}^{t-1}(1-\alpha {)}^j{X}_{t-j}+(1-\alpha {)}^t{\ell }_0$, so older observations receive exponentially decreasing weights. Forecast ${\hat{X}}_{t+h\mid t}={\ell }_t$ for every $h\ge 1$. This is appropriate for a locally stable level with no persistent trend or seasonality.

For a local linear trend, double exponential smoothing supplies level and slope. Holt's linear trend method usesAnother standard convention is Brown's double exponential smoothing: smooth twice with the same gain, $s_t^{(1)}=\alpha X_t+(1-\alpha )s_{t-1}^{(1)}$ and $s_t^{(2)}=\alpha s_t^{(1)}+(1-\alpha )s_{t-1}^{(2)}$, then put $a_t=2s_t^{(1)}-s_t^{(2)}$, $b_t=\alpha (s_t^{(1)}-s_t^{(2)})/(1-\alpha )$ and forecast $a_t+h{b}_t$. These variants handle a nonseasonal trend; seasonal effects need separate modelling, and a long-horizon linear extrapolation assumes the current local slope persists.

For a series with constant mean $\mu$, write $X_t=x_t-\mu$. An autoregressive moving-average process has the formwhere $w_t$ is zero-mean white noise with variance ${\sigma }_w^2>0$. The backshift operator is $B{X}_t=X_{t-1}$. Define the autoregressive polynomial and the moving-average polynomial byThe autoregressive operator and moving-average operator give the concise equationOrders are normally specified minimally, with nonzero last coefficients and no common polynomial factor. A displayed unreduced equation can have larger orders than the process actually needs.

Causality means that the stationary process can be recovered from present and past noise:Invertibility means that the noise can be recovered from present and past observations:The causality and invertibility root criteria for an ARMA model, for the reduced representation, areIndeed the transfer series are $\sum {\psi }_j{z}^j=\Theta (z)/\Phi (z)$ and $\sum {\pi }_j{z}^j=\Phi (z)/\Theta (z)$. When all denominator roots lie outside the closed unit disc, each rational function is analytic on a larger disc, so its Taylor coefficients decay geometrically and are absolutely summable. Conversely, a genuine denominator zero in the closed unit disc gives a pole incompatible with such a convergent filter. The reduced-representation proviso prevents a cancelled root from being mistaken for an obstruction.

For order identification by autocorrelation cutoffs, first estimate the sample autocorrelation function and sample partial autocorrelation function from a suitably stationary, centred series. The autocorrelation function is $\rho (h)=\mathrm{Cov}(X_t,X_{t-h})/\mathrm{Var}(X_t)$. The lag-$h$ partial autocorrelation function is the last coefficient in the best linear regression of $X_t$ on $X_{t-1},\dots ,X_{t-h}$, equivalently the correlation left after removing the intervening lags.

For a pure moving-average model of order $q$, observations more than $q$ lags apart share no noise, so the population ACF is zero beyond $q$, while the PACF usually tails off. For a pure autoregressive model of order $p$, the PACF is zero beyond $p$, while the ACF usually decays, possibly with damped oscillation. Thus a clear sample ACF cutoff suggests an MA order and a clear PACF cutoff suggests an AR order. For a mixed ARMA process both usually tail off, so these plots guide a small set of candidate orders rather than determine $p,q$ uniquely. Sampling variation blurs cutoffs; compare fitted candidates and check that their residual ACF resembles white noise. Common-factor cancellation can also disguise the orders in a displayed equation.

Use the sample autocorrelation function and sample partial autocorrelation function to look for approximate cutoffs. For a minimal causal autoregressive model of order $p$, the partial autocorrelation function is zero beyond $p$, while the autocorrelation function usually tails off, possibly with damped oscillation. For an invertible moving-average model of order $q$, the autocorrelation function is zero beyond $q$, while the partial autocorrelation function tails off. In a mixed autoregressive moving-average model, both generally tail off.

Thus the last visibly nonzero partial correlation suggests an autoregressive order, and the last visibly nonzero autocorrelation suggests a moving-average order. Finite samples do not produce exact zeros. Approximate white-noise reference bands can help flag clearly nonzero lags, but they are not universal confidence intervals for arbitrary correlated processes. Fit a few suggested orders and check residual autocorrelation, rather than treating these plots as a proof of the model order. This is order identification by autocorrelation cutoffs.