Past exam of the mathematics course of the University of Cambridge / 2004 / iii / Paper 40 / 1

For a weakly stationary process with mean $m$ and nonzero variance, put $\gamma (h)=\mathrm{Cov}(X_{t+h},X_t)$ and $\rho (h)=\gamma (h)/\gamma (0)$. The function $\rho$ is its autocorrelation function. For observations $x_1,\dots ,x_N$, one common sample autocorrelation function isA correlogram plots these estimates against lag; the population analogue plots $\rho (h)$.

The lag-$h$ partial autocorrelation function measures the correlation between $X_t$ and $X_{t-h}$ after removing their linear projections on the intervening observations. Its sample counterpart can be calculated by the Yule-Walker equations: solveand take $\hat{\alpha }(h)={\hat{\varphi }}_{h,h}$, assuming the matrix is nonsingular. This defines a standard sample partial autocorrelation function.

For a minimal causal autoregressive model of order $p$, the autocorrelation typically decays geometrically, possibly with damped oscillations, whereas the partial autocorrelation function is zero beyond lag $p$: after those $p$ preceding values have been projected out, the innovation is orthogonal to older values. For a moving-average model of order $q$, nonoverlapping sets of driving white noise give $\gamma (h)=0$ for $|h|>q$; its partial autocorrelation function generally tails off when the representation is invertible. Thus an ACF cutoff suggests an MA order; a PACF cutoff suggests an AR order. An ARMA model generally has neither cutoff. These are population properties; sample noise makes cutoffs approximate, so fitted residuals and uncertainty should also be inspected. Under independent white noise, fixed-lag sample correlations have approximate standard error $N^{-1/2}$, giving a rough diagnostic band rather than a universal simultaneous test.

For a discrete-time weakly stationary process, the spectral density of a stationary process, when it exists, is a nonnegative function satisfyingIf ${\sum }_h|\gamma (h)|<\infty$, thenA general second-order stationary process has a spectral measure of a stationary time series, which need not have a spectral density: a centered random constant has an atom at zero. Thus the spectral density calculation below assumes that $f_X$ exists; the corresponding measure identity holds without that extra assumption.

Absolute summability of the filter coefficients gives mean-square convergence becauseConsequently the bilateral filtered process is well defined, with constant mean $m{\sum }_s{a}_s$. Its covariance isThe sum is absolutely convergent, since $|{\gamma }_X(j)|\le {\gamma }_X(0)$ and $\sum |a_s|<\infty$. It depends only on lag, proving weak stationarity of $Y$.

Insert the spectral representation and interchange summation and integration using absolute summability and the finite total spectral mass. For real coefficients,Therefore the spectral density transformation under a linear filter isFor a bilateral sequence, $A$ is guaranteed to converge on the unit circle. Its negative powers need not converge inside the disk: for example, $a_{-j}=2^{-j}$ makes the series diverge at $z=1/4$. Only the unit-circle values are used here, so the unnecessarily broad domain printed for $A$ does not affect the proof.

Center the five coefficients at lags $-2,-1,0,1,2$. Their transfer function is real:Choosing consecutive causal lags instead only adds a phase and leaves the squared modulus unchanged. After $k$ applications, the white noise spectral density has becomeWrite $c=\cos \omega$. Completing the square givesFor $-1\le c\le 1$, its range is $[-1/3,7/6]$, and its absolute maximum is $7/6$, attained exactly when $c=1/2$. Consequently, on the conventional nonnegative frequency range $[0,\pi ]$,On the two-sided range $[-\pi ,\pi ]$ there is also a peak at $-\pi /3$; the ratio there is one for every $k$. This is a counterexample to the claim if read literally on a signed frequency domain. The correct two-sided exclusion is $\omega \ne \pm \pi /3$.

The result illustrates how repeated symmetric filtering can amplify an oscillation. Although $H(0)=1$ preserves a constant component, $H(\pi /3)=7/6>1$ amplifies period-six oscillations. Repetition selects an increasingly narrow frequency band around those peaks; it does not simply smooth all noise away. In fact the unnormalized output variance eventually grows without bound, since a fixed interval around either peak has $|H|>1$.