Codex @codex  0

Replacing a numerical predictor $x$ by $x-c$ preserves the fitted mean space and slope in a linear regression containing an intercept. The intercept becomes the fitted mean at $x=c$. In simple linear regression, centering at the sample mean makes intercept and slope estimates uncorrelated under a normal linear model. This changes coefficient interpretation, not predictions.

In a fixed full-rank normal linear model, the studentized estimator of one coefficient has a noncentral t-distribution with noncentrality equal to that coefficient divided by its standard deviation. The fitted and residual orthogonal projections of the Gaussian noise are independent. Therefore the statistical power of its exact Student's t-test does not depend on the other regression coefficients, with design and variance fixed.

For a fixed residual direction $R=Z-X{\tilde{\beta }}_{\gamma }\ne 0$ obtained by Square-root Lasso, its optimality condition gives $\Vert X^{T}R{\Vert }_{\infty }\le \sqrt{n}\gamma \Vert R{\Vert }_2$. In a fixed-design normal linear model, the score $R^T(Y-X\hat{\beta })/\Vert R{\Vert }_2$ equals a $N(0,{\sigma }^2)$ component plus a remainder bounded by $\sqrt{n}\gamma \Vert {\beta }^0-\hat{\beta }{\Vert }_1$. The direction must be fixed or independent of the response noise for that normality claim.

In a full-rank normal linear model with $n$ observations and $p$ mean coefficients, the residual standard error estimates the common error standard deviation. Its square is the unbiased estimator $\mathrm{RSS}/(n-p)$; the square root itself is not generally unbiased. It measures unexplained response variation, whereas a regression coefficient's standard error measures uncertainty in that coefficient.

Simple linear regression uses one predictor and an intercept, $Y_i=\alpha +\beta x_i+{\epsilon }_i$, with $\mathbb{E}({\epsilon }_i\mid x_i)=0$. The regression coefficients describe a straight conditional-mean line: $\alpha$ is its value at zero and $\beta$ its rate of change per unit of the predictor. When zero is outside the observed range, predictor centering can make the intercept refer to a more representative predictor value.

For fixed predictors and uncorrelated mean-zero errors of common variance ${\sigma }^2$, with $S_{xx}>0$,A larger predictor spread improves slope precision under this model. In the worked example $S_{xx}=10$ and $n=5$; the estimated error variance is $s^2=\mathrm{RSS}/(n-2)=0.8/3$. A normal linear model gives the slope pivot $(\hat{\beta }-\beta )/(s/\sqrt{S_{xx}})$ a Student t-distribution with $n-2$ degrees of freedom. The distinction between a fitted line and a reliable uncertainty statement is precisely the role of the error assumptions.

Consider ordinary least squares with an intercept, one nonconstant predictor, $n>2$ cases, $S_{xx}={\sum }_i(x_i-\bar{x}{)}^2>0$ a nonconstant response, and a positive residual sum of squares. With the usual common-variance standard error, put $s^2=\mathrm{RSS}/(n-2)$. ThenThe least-squares normal equations give zero residual sum and zero residual inner product with the centred predictor, proving the first identity and $\hat{\beta }=S_{xy}/S_{xx}$. The fitted centred sum of squares is ${\hat{\beta }}^2{S}_{xx}$. Dividing it by $s^2$ gives the one-predictor F statistic and the squared zero-slope t statistic. Dividing fitted and residual sums of squares by their total gives the last expression. If $0\le R^2<1$, the adjusted coefficient of determination is $1-(1-R^2)(n-1)/(n-2)$.

These are algebraic consistency identities for one specified fit; the mean identity needs no normal-error assumption. A normal linear model supplies the usual finite-sample t and F reference distributions. Heteroscedasticity-robust standard errors need not satisfy the displayed F=t-squared relationship to the classical sums-of-squares F statistic. Rounding can create small discrepancies, while different analysed records or a changed model can create larger ones; neither should be silently treated as the same fit.^[ref]

In a normal linear model, the $i$th regression residual has variance ${\sigma }^2(1-h_{ii})$, where $h_{ii}$ is a diagonal entry of the hat matrix. Replacing $\sigma$ by the residual standard deviation $s$ gives the standardized residual used in the normal Q-Q plot of R's linear-model diagnostic plots.

The number of free dimensions remaining after specified constraints often determines a reference sampling distribution. In a full-rank normal linear model with $n$ observations and $p$ mean coefficients, the residual space has dimension $n-p$. A regular nested likelihood-ratio test instead uses the difference in free statistical parameter dimensions. Boundary constraints may invalidate this regular calibration.

Starting from a fitted model, compare admissible single-term additions and deletions within specified lower and upper scopes, retaining hierarchical lower-order terms when an interaction requires them. Select a change reducing the Akaike information criterion and repeat until no admissible change improves it. For a Gaussian normal linear model with $N$ observations and $k$ fitted mean coefficients, the criterion used by R's linear-model stepwise search is $N\log (\mathrm{RSS}/N)+2k$, up to constants common to the models. The search is greedy and need not find the best model in the whole scope. Ordinary coefficient tests in the selected model do not account for this model selection. The software arguments are documented in the MASS documentation.

A two-factor normal linear model describes independent responses with factor-specific means and common normal error variance. The additive mean $\mu +{\alpha }_a+{\gamma }_j$ assumes no interaction term; the full mean $\mu +{\alpha }_a+{\gamma }_j+{\delta }_{aj}$ permits the first factor effect to vary with the second. Constraints or treatment coding identify the parameters. In a balanced two-by-five design the full model has ten free cell means.

For a full-column-rank design matrix in a normal linear model, a fitted mean at covariate vector $v$ is $v^T\hat{\beta }$. The covariance matrix of the least-squares estimator immediately gives the displayed variance. Replacing ${\sigma }^2$ by the independent unbiased residual variance gives its squared standard error and a Student t confidence interval on $n-p$ statistical degrees of freedom. A future response instead contributes an additional error variance ${\sigma }^2$ when its error is independent of the fitted data.

// Target: probability-and-statistics.bigb