Best linear prediction from an infinite past (${\hat{X}}_{t+1}={\mathrm{proj}}_{{\mathcal{H}}_t}{X}_{t+1}$)

The best mean-square linear predictor from a semi-infinite past is the orthogonal projection onto the closed linear span of past observations in the random-variable Hilbert space. For a causal invertible ARMA representation, past observations and past innovations generate the same closed span. The next linear innovation is orthogonal to it and is therefore the prediction error.