= Finite-past innovation
{title2=$U_t=X_t-\operatorname{proj}_{\operatorname{span}(X_1,\ldots,X_{t-1})}X_t$}
For centered square-integrable <random variables> $X_1,\ldots,X_T$, their <finite-past innovations> are the successive errors of <best linear prediction from a finite past>:
$$
U_1=X_1,\qquad
U_t=X_t-\operatorname{proj}_{\operatorname{span}(X_1,\ldots,X_{t-1})}X_t.
$$
Here the <inner product> is $\langle X,Y\rangle=\mathbb E[XY]$ in the <Hilbert space> of square-integrable <random variables>. By the defining property of <orthogonal projection>, $U_t$ is orthogonal to every earlier observation, hence to every earlier <innovation>. Recursively, each $X_t$ is $U_t$ plus a <linear combination> of earlier <innovations>. Thus
$$
\operatorname{span}(X_1,\ldots,X_t)=\operatorname{span}(U_1,\ldots,U_t).
$$
This equality remains true when some <innovations> vanish. Only the nonzero <innovations> form an <orthogonal basis>: if $X_2=X_1$, then $U_2=0$, and a formula dividing by $\operatorname{var}(U_2)$ is invalid. When all <innovation> <variances> are positive, the construction is the <Gram-Schmidt process> applied to the observations. <Finite-past innovations> depend on the starting time; unlike <innovations> from <best linear prediction from an infinite past>, they do not use observations before initialization. Additional stability assumptions are needed to assert that this initialization effect disappears.
Back to article page