State-space model (time series, $S_t=F{S}_{t-1}+Z_t,Y_t=H{S}_t+W_t$)

A state-space model separates a latent transition equation $S_t=F{S}_{t-1}+Z_t$ from an observation equation $Y_t=H{S}_t+W_t$. A complete specification includes the initial-state law and its relation to the noise sequences, not just the two equations. Linear second-order models specify covariance structure; Gaussian versions additionally prescribe joint Gaussian laws.

For an autoregressive moving-average model of order $(1,1)$, this state has observation row $F=(\varphi ,1,\theta )$ and transition matrixThe state noise is $(0,{\epsilon }_t,0{)}^{\top }$. Its first transition row reconstructs $X_{t-1}$ and its third row shifts the previous white noise value.

For $|\varphi |<1$ and centered independent white noise values, the state above has expected value zero and this stationary covariance matrix, where ${\gamma }_0={\sigma }^2(1+{\theta }^2+2\varphi \theta )/(1-{\varphi }^2)$. The cross-covariance with the future white noise value is zero and that with the contemporaneous white noise value is ${\sigma }^2$. Direct multiplication verifies $P=GP{G}^{\top }+Q$.

The latent state follows a random walk, while observations add independent measurement noise. If the two noises have variances $W,V$, the differences have autocovariances $W+2V$ at lag zero, $-V$ at lag one, and zero at larger lags. For $V,W>0$, write $q=(W+2V+\sqrt{W^2+4WV})/2$ and $\vartheta =-V/q$. The differenced process is an invertible moving-average model with coefficient $\vartheta$ and innovation variance $q$. Its level has a unit root.

In a linear Gaussian state-space model, let the predicted state mean and covariance be $m_t^{-},P_t^{-}$. The observation innovation process has conditional mean zero and covariance ${\Sigma }_t=A{P}_t^{-}{A}^{\top }+R$, with innovation ${ϵ}_t=y_t-A{m}_t^{-}$. The conditional multivariate normal distribution gives the predictive observation density. Multiplying these conditional densities by the chain rule gives the displayed likelihood function; equivalently the innovations are independent Gaussian vectors. The Kalman filter supplies $m_t^{-},P_t^{-}$ recursively, so one evaluates the exact observed-data likelihood function without integrating all latent states jointly.

For $|\varphi |<1$, initialize the state by $S_0={\sum }_{j\ge 0}{\varphi }^j{Z}_{-j}$, with variance ${\sigma }_z^2/(1-{\varphi }^2)$. It is orthogonal to future state noise and to observation noise orthogonal to all state noise. Under a Gaussian specification, take the initial state Gaussian with this variance and independent of future noises. Starting at zero instead gives a transient model.