Ordinary least squares estimators ($\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$)

For a linear regression design matrix $X$ with full column rank, the ordinary least squares coefficient vector is $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$. In a normal linear model with error covariance matrix ${\sigma }^{2}I$, it is an unbiased estimator with covariance matrix ${\sigma }^2(X^{T}X{)}^{-1}$. Replacing ${\sigma }^2$ by the residual variance estimate gives the reported coefficient standard errors.

In simple linear regression with a nonconstant predictor,

For a nonconstant predictor, putThe minimized residual sum of squares isEach centered sum can be recovered in constant time from the five raw sums $\sum x_i$, $\sum y_i$, $\sum x_i^2$, $\sum x_i{y}_i$, and $\sum y_i^2$.