Innovation process

An innovation measures the part of the current observation not predicted from past observations. The linear innovation process uses orthogonal projection onto the past linear span. A conditional expectation predictor can use nonlinear information; the two predictors agree for a Gaussian process.

Observe $Y_t=\lambda t+W_t$, with an independent Gaussian random vector $\lambda$ of mean $m_0$ and covariance matrix $V_0$. Its posterior is Gaussian with mean $m_t=(I+t{V}_0{)}^{-1}(m_0+V_0{Y}_t)$ and covariance matrix $V_t=V_0(I+t{V}_0{)}^{-1}$. These formulas also hold for a singular positive semidefinite matrix $V_0$. When $V_0$ is invertible, completing the square in the Bayes' theorem gives $V_t^{-1}=V_0^{-1}+tI$ and $m_t=V_t(V_0^{-1}{m}_0+Y_t)$. The observed innovation process ${\hat{W}}_t=Y_t-{\int }_0^t{m}_{s}ds$ is a Brownian motion in the observation filtration, and $d{m}_t=V_{t}d{\hat{W}}_t$.

For the Gaussian Brownian drift filter with positive definite $V_0$, define $Z_t=\det (I+t{V}_0{)}^{1/2}\exp [-m_t^T{V}_t^{-1}{m}_t/2+m_0^T{V}_0^{-1}{m}_0/2]$. Because $V_t^{\prime }=-V_t^2$ and $d{m}_t=V_{t}d{\hat{W}}_t$, the Itô formula gives $d\log Z_t=-m_t^{T}d{\hat{W}}_t-|m_t{|}^{2}dt/2$. Thus $Z$ is a positive stochastic exponential, hence a nonnegative local martingale and a supermartingale. For a singular prior, the inverse-matrix expression is undefined; the same process can instead be defined by $Z_t=({\mathbb{E}}_{\lambda \sim N(m_0,V_0)}\exp [{\lambda }^T{Y}_t-t|\lambda {|}^2/2]{)}^{-1}$, with $Y_t$ treated as fixed inside this Gaussian integral.

For the Gaussian Brownian drift filter $Y_t=\alpha t+W_t$, with independent prior $\alpha \sim N(m_0,{\tau }_0^{-1})$, the observation law has densityrelative to driftless Wiener measure. The law having fixed observation drift $k$ has density $e^{k{Y}_t-k^{2}t/2}$ relative to the same reference. Their ratio is therefore a positive mean-one density on every finite horizon. In the observation filtration it satisfies $d{D}_t=-D_t(m_t-k)d{\hat{W}}_t$, where $m_t$ is the posterior mean and $\hat{W}$ the innovation process. This density-ratio argument establishes a true martingale without an unsupported global Novikov condition for the unbounded posterior mean.

For $Y_s=\lambda s+W_s$ and fixed $t>0$, subtracting $(s/t)Y_t$ leaves the Brownian bridge $W_s-(s/t)W_t$. It is independent of $(\lambda ,Y_t)$, since it is independent of $\lambda$ and has zero covariance with $W_t$ in their jointly Gaussian law. The observed path is determined by this bridge and its endpoint, so the bridge gives no further information about $\lambda$. Hence conditioning on the full observation filtration at $t$ gives the same posterior as conditioning on $Y_t$ alone.

For $X_t=W_t+\alpha t$ with an independent equiprobable drift $\alpha \in \{-a,a\}$, the likelihood ratio between the two drifts is $e^{2a{X}_t}$. Multiplying the normal distribution densities of successive increments proves this ratio for every observation partition; continuity then gives the full observed-path posterior. The Bayes' theorem gives $q_t=\mathbb{P}(\alpha =a\mid {\mathcal{F}}_t^X)=(1+e^{-2a{X}_t}{)}^{-1}$. Thus the observed drift is $a(2q_t-1)=a\tanh (a{X}_t)$. The process ${\hat{W}}_t=X_t-{\int }_0^{t}a\tanh (a{X}_s)ds$ is an observed-filtration martingale with quadratic variation $t$, so the Lévy characterization of Brownian motion makes it a Brownian motion. The observed state is a scaled diffusion with hyperbolic tangent drift.

For a square-integrable time series, let ${\mathcal{H}}_{t-1}$ be the closed linear span of its past. The innovation ${\epsilon }_t=X_t-{\mathrm{proj}}_{{\mathcal{H}}_{t-1}}{X}_t$ is orthogonal to every past linear observation. A causal invertible autoregressive moving-average model is driven by these innovations, which are weak white noise for a weakly stationary process.