A linear least-squares projection is the affine predictor that minimizes mean squared error. For real random variables with finite second moments and , minimizing givesThe residual satisfies and . Consequently it is orthogonal, in the inner product , to the span of the constant variable and . For any other affine predictor , expanding the square yieldsbecause the cross term vanishes. This proves optimality and uniqueness of the predicted random variable. The residual variance is the Schur complement covariance. No normal distribution is needed. For several predictors, the same orthogonality argument gives the normal equations for Linear least squares; a singular predictor covariance can make coefficients nonunique while leaving the projected prediction unique.
For finite second moments and , the residual variance after affine projection onto one predictor is , a covariance Schur complement.
Articles by others on the same topic
There are currently no matching articles.