Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 37 / 3

For a centered weakly stationary process with $\gamma (0)>0$, the autocorrelation function isFor $h\ge 2$, let $P$ be orthogonal projection in L2 space onto the linear span of $X_{t-1},\dots ,X_{t-h+1}$. The lag-$h$ partial autocorrelation function is the correlation coefficient of $X_t-P{X}_t$ and $X_{t-h}-P{X}_{t-h}$. It removes the linear contribution of the intervening observations. At lag one it is just $\rho (1)$. For nonsingular prediction covariance matrices, it is equivalently the last coefficient $a_{hh}$ in the order-$h$ linear predictor of $X_t$ from $X_{t-1},\dots ,X_{t-h}$. Equal residual variances, by stationarity, identify that coefficient with the residual correlation. If a residual has zero variance, this correlation is undefined; the nondegeneracy condition matters.

Use the sample autocorrelation function and sample partial autocorrelation function to look for approximate cutoffs. For a minimal causal autoregressive model of order $p$, the partial autocorrelation function is zero beyond $p$, while the autocorrelation function usually tails off, possibly with damped oscillation. For an invertible moving-average model of order $q$, the autocorrelation function is zero beyond $q$, while the partial autocorrelation function tails off. In a mixed autoregressive moving-average model, both generally tail off.

Thus the last visibly nonzero partial correlation suggests an autoregressive order, and the last visibly nonzero autocorrelation suggests a moving-average order. Finite samples do not produce exact zeros. Approximate white-noise reference bands can help flag clearly nonzero lags, but they are not universal confidence intervals for arbitrary correlated processes. Fit a few suggested orders and check residual autocorrelation, rather than treating these plots as a proof of the model order. This is order identification by autocorrelation cutoffs.

For the moving-average process of order one, $\gamma (0)={\sigma }^2(1+{\theta }_1^2)$, $\gamma (1)={\sigma }^2{\theta }_1$, and $\gamma (2)=0$. Regressing each endpoint on the single intervening observation leaves residual covariance $\gamma (2)-\gamma (1{)}^2/\gamma (0)$ and residual variance $\gamma (0)-\gamma (1{)}^2/\gamma (0)$. Therefore its lag-two partial autocorrelation function isThe denominator is positive. In particular, the moving-average autocorrelation cutoff at lag one does not make the lag-two partial correlation zero unless ${\theta }_1=0$.

Use cycles-per-time spectral density, with frequency $\omega \in [-1/2,1/2]$. The spectral representation theorem for a stationary time series gives the centered white noise representationwhere disjoint increments are orthogonal. Put ${\theta }_0=1$. Since the filter is finite, substitute each noise representation and interchange the finite sum with the integral:The new orthogonal increment measure is $d{Z}_X=\Theta (e^{-2\pi i\omega })d{Z}_{\epsilon }$. Its variance measure is therefore ${\sigma }^2|\Theta (e^{-2\pi i\omega }){|}^{2}d\omega$. The coefficients are real, so conjugation changes the sign of the exponent without changing the modulus. HenceWith angular frequency $\lambda =2\pi \omega$, the spectral density of a stationary process instead contains the factor $1/(2\pi )$. The convention explains its absence here. Invertibility is not needed for this finite-filter spectral calculation.

The autocovariance vanishes beyond lag $q$. For an ordinary consecutive sum, variance of a sum givesThe last sum is finite. For $h\ge 0$, $\gamma (h)={\sigma }^2{\sum }_{j=0}^{q-h}{\theta }_j{\theta }_{j+h}$. Counting all coefficient pairs givesFor the odd-indexed sample the lag-$h$ autocovariance is $\gamma (2h)$, so the same finite-sum argument gives ${\sum }_h\gamma (2h)$. Only coefficient pairs of the same parity contribute. If $E={\sum }_{j\mathrm{even}}{\theta }_j$ and $O={\sum }_{j\mathrm{odd}}{\theta }_j$, the odd-subsample limit isThis is the odd-subsample long-run variance of a moving average. Equivalently it is $[f_X(0)+f_X(1/2)]/2$ in the cycles convention. Under the printed invertibility assumption and positive noise variance, the polynomial is nonzero at both $1$ and $-1$, so both limits are positive. The same formulas also hold without invertibility, when cancellations can make a limit zero.