Finite-past innovation Created 2026-10-07 Updated 2026-11-05
For centered square-integrable random variables , their finite-past innovations are the successive errors of best linear prediction from a finite past:
Here the inner product is in the Hilbert space of square-integrable random variables. By the defining property of orthogonal projection, is orthogonal to every earlier observation, hence to every earlier innovation. Recursively, each is plus a linear combination of earlier innovations. Thus
This equality remains true when some innovations vanish. Only the nonzero innovations form an orthogonal basis: if , then , and a formula dividing by is invalid. When all innovation variances are positive, the construction is the Gram-Schmidt process applied to the observations. Finite-past innovations depend on the starting time; unlike innovations from best linear prediction from an infinite past, they do not use observations before initialization. Additional stability assumptions are needed to assert that this initialization effect disappears.
The autocovariance of an MA(1) process comes directly from shared noise terms:
Work with the zero-mean white noise convention and . Put . Suppose the preceding finite-past innovations are mutually orthogonal. Their triangular relation to the observations means that they span the same finite past. For , is a linear combination of , so . Moreover,
Projection onto this orthogonal basis therefore has only its last coefficient nonzero:
The residual is orthogonal to the entire preceding span, proving the induction. Expanding its variance gives
The initialization is , , and . Each finite covariance matrix is a positive-definite matrix: the last observation in any nonzero finite linear combination contains a noise term absent from earlier observations. Thus all the projection denominators are positive. These are the finite-sample innovations of an MA(1) process, requiring no Gaussian assumption.
Substituting the covariances into the recursion gives
The denominator has a positive limit when . Taking limits gives . The permitted interval selects
For , every coefficient is zero; at , the roots coincide. This is the limiting MA(1) innovations coefficient. The limiting innovation variance is correspondingly .
For the observed numerical case, and . The exact innovation calculations are
Therefore the predictor of based on the two observations and its error variance are
For , the available information is still only . Since its covariances with both and are zero, its orthogonal projection onto that span is zero:
This is two-step prediction for an MA(1) process. Applying the next one-step recursion would instead use an observed , which is not available here.