Consistency of least squares Created 2026-09-24 Updated 2026-11-05
For fixed coefficient dimension and , the ordinary least squares error satisfies
Consequently convergence in probability of to a positive-definite matrix , together with in probability, proves statistical consistency by continuity of matrix inversion.
For independent and identically distributed random-design pairs, finite second moments and imply these limits by the law of large numbers, provided is nonsingular. Conditional mean zero, , is a familiar sufficient exogeneity condition. For a deterministic design matrix, the vanishing normalized design-error product must instead be justified from its errors and weights.
Finite moments and a nonsingular design alone are insufficient. In an intercept-only model with errors identically one, the estimated intercept is always . The missing zero-expected value condition changes the limiting target.