Simple linear regression uses one predictor and an intercept, , with . The regression coefficients describe a straight conditional-mean line: is its value at zero and its rate of change per unit of the predictor. When zero is outside the observed range, predictor centering can make the intercept refer to a more representative predictor value.
For fixed predictors and uncorrelated mean-zero errors of common variance , with ,A larger predictor spread improves slope precision under this model. In the worked example and ; the estimated error variance is . A normal linear model gives the slope pivot a Student t-distribution with degrees of freedom. The distinction between a fitted line and a reliable uncertainty statement is precisely the role of the error assumptions.
Consider ordinary least squares with an intercept, one nonconstant predictor, cases, a nonconstant response, and a positive residual sum of squares. With the usual common-variance standard error, put . ThenThe least-squares normal equations give zero residual sum and zero residual inner product with the centred predictor, proving the first identity and . The fitted centred sum of squares is . Dividing it by gives the one-predictor F statistic and the squared zero-slope t statistic. Dividing fitted and residual sums of squares by their total gives the last expression. If , the adjusted coefficient of determination is .
These are algebraic consistency identities for one specified fit; the mean identity needs no normal-error assumption. A normal linear model supplies the usual finite-sample t and F reference distributions. Heteroscedasticity-robust standard errors need not satisfy the displayed F=t-squared relationship to the classical sums-of-squares F statistic. Rounding can create small discrepancies, while different analysed records or a changed model can create larger ones; neither should be silently treated as the same fit.[ref]
Summarizing Linear Model Fits in the R documentation. Value: coefficient matrix, sigma, df, fstatistic and r.squared
In simple linear regression written as , the constant and centred predictor columns are orthogonal. Differentiating the sum of squared residuals gives the displayed ordinary least squares estimators, provided . For independent mean-zero errors of common variance , both estimators are unbiased, their variances are and , and their covariance is zero. The uncentred intercept is .
Articles by others on the same topic
Simple linear regression is a statistical method used to model the relationship between two continuous variables by fitting a linear equation to the observed data. It assumes that there is a linear relationship between the independent variable (predictor) and the dependent variable (response). ### Key Components of Simple Linear Regression: 1. **Independent Variable (X)**: This is the variable that you use to predict the value of the dependent variable. It is also known as the predictor, feature, or explanatory variable.