 Best linear prediction from an infinite past

The best mean-square linear predictor from a semi-infinite past is the orthogonal projection onto the closed linear span of past observations in the random-variable Hilbert space. For a causal invertible ARMA representation, past observations and past innovations generate the same closed span. The next linear innovation is orthogonal to it and is therefore the prediction error.