Past exam of the mathematics course of the University of Cambridge / 2018 / iii / Paper 218 / 5

Use the stationary autoregressive process of order oneMultiplication by $X_{t-1}-\mu$ and expectation, using independence of the innovation from the past, gives the autocovariance equation $\gamma (1)=\varphi \gamma (0)$. With $\bar{X}=n^{-1}{\sum }_t{X}_t$, define the sample autocovariances using the same denominator:Substitution into that equation gives the Yule–Walker estimator for an autoregressive process of order oneThis is exactly the lag-one sample autocorrelation function under the divisor-$n$ convention, provided the sample has positive variance. Using different divisors for different lags would change the estimator.

The parameters are $\mu \in \mathbb{R}$, $|\varphi |<1$ and innovation variance ${\sigma }^2>0$. Stationarity gives $X_1\sim N(\mu ,{\sigma }^2/(1-{\varphi }^2))$, and the Markov property gives $X_t\mid X_{t-1}\sim N(\mu +\varphi (X_{t-1}-\mu ),{\sigma }^2)$. Multiplying the initial density and the transition densities yields the stationary Gaussian AR1 likelihood, with $n=31$:The initial-observation factor depends on all three parameters and must be retained for the full stationary likelihood function. Omitting it gives a conditional likelihood given $X_1$, a different objective.

By the Markov property, all factors in the stationary likelihood function that involve $X_2$ are the transitions from $X_1$ to $X_2$ and from $X_2$ to $X_3$. Hence conditioning on later observations adds no information once the two neighbors are known. Writing $u=X_2-\mu$, $u_1=x_1-\mu$ and $u_3=x_3-\mu$, its conditional density is proportional toThe quadratic coefficient of $u$ is $1+{\varphi }^2$ and the linear coefficient is $-2\varphi (u_1+u_3)$. Completing the square therefore gives the Gaussian AR1 bridgeHere ${\sigma }^2$ is the innovation variance, not the marginal stationary variance.

Apply EM for a missing observation in a Gaussian AR1 process, starting from ${\theta }^{old}=({\mu }^{old},{\varphi }^{old},({\sigma }^2{)}^{old})$ in the stationary parameter space. The Gaussian AR1 bridge gives the E-step quantitiesThus $\mathbb{E}[X_2\mid x_{obs}]=m$, $\mathbb{E}[X_2^2\mid x_{obs}]=m^2+v$, and $\mathbb{E}[X_1{X}_2\mid x_{obs}]=x_{1}m$, $\mathbb{E}[X_2{X}_3\mid x_{obs}]=m{x}_3$.

Let $x_2^{\ast }=m$ and $x_t^{\ast }=x_t$ otherwise. Keeping $m,v$ fixed throughout the M-step, defineThe last term comes from the missing observation's conditional variance in the two adjacent innovation squares. The expected complete-data log-likelihood isFor fixed $\varphi$, differentiating the quadratic in $\mu$ givesAlso the maximizing variance is ${\sigma }^2=R(\mu ,\varphi )/31$. The M-step is therefore a one-dimensional profile likelihood maximization:Repeat the E- and M-steps until the observed-data likelihood and parameters stabilize. Exact M-steps make that likelihood nondecreasing; different initializations help detect different local optima, and convergence alone is not a guarantee of a global maximum. At an interior local maximum, impute the missing temperature by its conditional mean $m$ evaluated at the fitted parameters, retaining $v$ as its conditional uncertainty. Replacing $X_2$ by $m$ and discarding $v$ would not implement the expectation-maximization algorithm.