Past exam of the mathematics course of the University of Cambridge / 2014 / iii / Paper 36 / 2

Interpret stationarity in the usual second-order time-series sense and assume nondegenerate noise, ${\sigma }_z^2>0$. For a two-sided autoregressive equation, the missing existence condition isIt is important to separate this from causality. If $|\varphi |<1$, the unique stationary solution is $X_t={\sum }_{j\ge 0}{\varphi }^j{Z}_{t-j}$. If $|\varphi |>1$, there is still a stationary solution, but it is anticausal:Both expansions converge in L2 because their coefficients are square summable. Substitution verifies the equation. Their means are zero and their covariance functions depend only on lag. Uniqueness follows by iterating the equation backward in the first case and forward in the second: the remainders ${\varphi }^n{X}_{t-n}$ or ${\varphi }^{-n}{X}_{t+n}$ tend to zero in L2 for any stationary finite-variance solution. This is the stationary versus causal solution of a two-sided AR(1) equation.

For $\varphi =\pm 1$, iteration givesThe variance of the right side is $n{\sigma }_z^2$. The variance of the left side is at most $4\mathrm{Var}(X_t)$ by stationarity and Cauchy-Schwarz inequality. These are incompatible as $n\to \infty$. Thus no weakly stationary finite-variance solution exists at those unit roots.

If the intended claim includes a causal innovation representation, its condition is instead $|\varphi |<1$, as in the next part. The stated white noise equation alone does not say that $Z_t$ is orthogonal to the past of $X$. If zero innovation variance is allowed, the unit-root exclusion has degenerate exceptions, such as random constant solutions when $\varphi =1$; the nondegenerate convention is necessary for the asserted nonexistence.

The condition for a causal linear-filter solution is $|\varphi |<1$. Its mean-square expansion is $X_t={\sum }_{j\ge 0}{\varphi }^j{Z}_{t-j}$. Summing the matching white noise terms givesThe assumed orthogonality of every $W_s$ to every $Z_t$ extends to every $X_t$ by L2 convergence of that expansion. Hence the added-noise process has mean zero andThis depends only on lag, proving weak stationarity. Its autocorrelation has the same geometric tail as the latent autoregression, but its positive-lag correlations are reduced by the additional variance at lag zero. This is the autocovariance of an AR(1) process observed with white noise. Strict stationarity or Gaussianity is not implied by white noise covariance assumptions alone.

Apply $1-\varphi B$ to the observed process and call the result $V_t$. ThenIts only nonzero covariance lags areSeek an invertible moving-average factor $V_t={\epsilon }_t+\vartheta {\epsilon }_{t-1}$ with $\mathrm{Var}({\epsilon }_t)=\nu$. Matching these covariances requires $\nu (1+{\vartheta }^2)=A$ and $\nu \vartheta =C$. Solving givesThese are the three requested parameters in the positive-sign moving-average convention. For nonzero total noise variance,so $|\vartheta |<1$ and the larger quadratic root gives the invertible factor.

Covariance matching alone would not identify arbitrary processes in distribution. To obtain an actual representation on the given space, defineThe series converges in L2. The spectrum of $V$ is $(A+2C\cos \omega )/(2\pi )=\nu |1+\vartheta e^{-i\omega }{|}^2/(2\pi )$, so this filtered process has constant spectrum $\nu /(2\pi )$ and is white noise. ThusThis is the invertible ARMA factorization of an AR(1)-plus-noise process. If ${\sigma }_w^2=0$, it reduces to AR(1); if $\varphi =0$, it reduces to white noise. If ${\sigma }_z^2=0$ and ${\sigma }_w^2>0$, then $\vartheta =-\varphi$, the common factor cancels and $Y=W$. Therefore the orders are at most (1,1); no unnecessarily minimal-order claim is made in those degenerate cases.

Take the latent autoregression as the scalar state $S_t$. A state-space model isThe transition and observation matrices are both scalar, $F=\varphi$, $H=1$. State-noise variance is $Q={\sigma }_z^2$, observation-noise variance is $R={\sigma }_w^2$, and the cross-noise covariance is zero at every pair of times.

A complete stationary initialization isIt is orthogonal to future state noise and to all observation noise. This is the stationary initialization of a scalar linear state-space model. This specifies the initial state in terms of the actual given two-sided white noise sequence, as well as its second-order law; simply starting from zero would give transient rather than stationary observations.

If a Gaussian state-space specification is intended, the complete specialization is $S_0\sim N(0,Q/(1-{\varphi }^2))$, independent of the future iid Gaussian state and observation noises, themselves independent with variances $Q,R$. Under the printed assumptions alone, Gaussian distributions and independence cannot be deduced from white noise orthogonality; the equations and stationary-series initialization above give the exact second-order representation without adding them.