Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 208 / 2

For a causal autoregressive model of order $p$, the population partial autocorrelation function cuts off after lag $p$, whereas the autocorrelation function generally decays. For an invertible moving-average model of order $q$, the autocorrelation function cuts off after lag $q$, whereas the partial correlation coefficients generally decay. For a mixed autoregressive moving-average model, both generally decay. Exponential decay may alternate in sign or show damped oscillations. Sample correlation coefficients only approximate these patterns, so isolated crossings of the significance bands are not exact order tests.

In PDF Figure 1, the autocorrelation function alternates sign, with a large negative lag-one correlation coefficient and geometrically decreasing magnitude. The partial autocorrelation function has essentially one substantial spike, negative at lag one and about $-0.8$; later values mostly lie within the displayed bands. Thus a stationary AR(1) with a negative coefficient is the natural first model, with $\hat{\varphi }$ initially near $-0.8$. The trace fluctuates around an approximately constant mean and shows no evident deterministic trend.

Fit that model, compare nearby low-order alternatives using likelihood function and an Akaike information criterion or Bayesian information criterion, and inspect the residual autocorrelation function and residual variance. A few small later PACF spikes are expected in a plot with many lags. The roughly $\pm 1.96/\sqrt{T}$ white noise bands are a guide, rather than simultaneous guarantees for every lag.

The sign alternation also suggests concentration of spectral power near the high-frequency end: the AR(1) denominator $1+{\varphi }^2-2\varphi \cos \lambda$ is smallest near $\lambda =\pi$ when $\varphi <0$. From estimated $\gamma (0)$ and lag-one correlation coefficient one can estimate innovation variance as ${\hat{\sigma }}_{\epsilon }^2\approx \hat{\gamma }(0)(1-{\hat{\varphi }}^2)$. The persistence parameter controls the decay rate and long-run variance of a stationary process. These plots do not establish a normal distribution, independence, or the absence of nonlinear dependence; correlation coefficients of squared residuals can provide a separate variance diagnostic.

With angular frequency $\lambda$, the periodogram isAt Fourier frequencies $2\pi j/T$ it is the squared modulus of the normalized discrete Fourier transform. If a nonzero mean is unknown, subtract the sample mean first.

For a zero-mean stationary process, expanding the square givesUnder absolute summability of the covariances this converges to the spectral density of a stationary process. Thus the periodogram is generally biased at finite $T$, but asymptotically unbiased under this short-memory time series condition.

It is nevertheless not a pointwise estimator with statistical consistency unless it is smoothed. For Gaussian white noise, at a nonzero Fourier frequency other than the Nyquist frequency, the real and imaginary Fourier components are independent normal variables. Exactly,where $f={\sigma }^2/(2\pi )$. The variance does not decrease with $T$. Under usual short-memory time series assumptions the same exponential limit is asymptotic for general processes. Averaging nearby frequencies or using a lag-window estimator reduces variance; a frequency bandwidth tending to zero while $T$ times that bandwidth tends to infinity can give statistical consistency. The raw plot remains useful for detecting strong periodic peaks, but increasing the record length alone does not remove its pointwise noise.

First, the absolute values in the printed limit are an error: the correct long-run variance of a stationary process is the signed sum of autocovariances. Directly,soEach coefficient tends to $1$ and has absolute value at most $1$. Absolute summability and the dominated convergence theorem therefore giveThe sum of absolute values is an upper bound, not the general limit. For a concrete counterexample to the printed assertion, take $X_t={\eta }_t-{\eta }_{t-1}$ with unit-variance iid noise. Then $\gamma (0)=2$, $\gamma (\pm 1)=-1$, and other covariances vanish. The absolute sum is $4$, but ${\overline{X}}_T=({\eta }_T-{\eta }_0)/T$, so $T\mathrm{Var}({\overline{X}}_T)=2/T\to 0$.

For correlated data the central limit theorem can still hold under suitable strong mixing of a stationary process and moment conditions, but the limiting variance is the long-run variance of a stationary process, rather than the one-observation variance. When it is positive,Positive serial dependence usually increases the standard error, while negative dependence can reduce it. The approximate effective sample size of a stationary sample is $T{\gamma }_X(0)/{\sum }_h{\gamma }_X(h)$ when that denominator is positive. A long-memory time series can require a different normalization or a different limit law. Absolute covariance summability alone does not prove a CLT: if $X_t=V{\eta }_t$ with a common independent random scale $V$ taking values $1$ and $2$ with equal probabilities and iid ${\eta }_t$ with the standard normal distribution, the off-diagonal covariances are zero, yet $\sqrt{T}{\overline{X}}_T$ has the nonnormal scale-mixture law $VN(0,1)$. The example is not an ergodic stationary process. If the long-run variance of a stationary process is zero, the usual nondegenerate square-root-$T$ CLT is unavailable.

For the causal AR(1), $|\varphi |<1$ andThe conditional distribution follows because the current innovation is independent of the past.

For a fixed known $\varphi$, condition on the observed $X_1$ and use the $n=T-1$ transitions $t=2,\dots ,T$. Their conditional maximum likelihood criterion is, up to constants,Differentiating in $\mu$ yieldsSince $X_t-\varphi X_{t-1}=(1-\varphi )\mu +{\epsilon }_t$,Thus it is an unbiased estimator, even conditionally on $X_1$, andIt has statistical consistency with mean-square convergence, and the iid-noise strong law of large numbers also gives almost-sure statistical consistency. Its conditional distribution is exactly normal with the displayed mean and variance. If $X_0$ is also observed and all $T$ transitions are used, replace $T-1$ by $T$.

The fixed-$\varphi$ qualification is necessary for the exact finite-sample claims. If $\varphi$ is jointly estimated, conditional likelihood function is linear regression with an intercept: writing ${\bar{X}}_{-}=n^{-1}{\sum }_{t=2}^T{X}_{t-1}$ and ${\bar{X}}_{+}=n^{-1}{\sum }_{t=2}^T{X}_t$, the unconstrained estimators areThis ratio is not generally an unbiased estimator and does not have the preceding finite-sample variance. Under the usual stationary regression conditions it has statistical consistency; its asymptotic variance is ${\sigma }_{\epsilon }^2/(1-\varphi {)}^2$. Indeed, it differs from ${\bar{X}}_{+}$ by $\hat{\varphi }(X_T-X_1)/(n(1-\hat{\varphi }))$, an asymptotically negligible endpoint term. Profiling an unknown innovation variance does not change the fixed-$\varphi$ estimate of $\mu$.