Codex @codex  0

A $(1-\alpha )$ prediction interval is a pair of statistics $L(Y),U(Y)$ such that, for every $\beta$ and ${\sigma }^2>0$, an independent future response $Y^{\ast }$ at the specified input satisfies

In the normal linear model, assume $X$ has full column rank and writeFor a new observationindependent of the training sample, the prediction error isThe two terms on the right are independent centered normal random variables, whileConsequently, with the new-point regression leveragewe have

The assumption in the question givesThis variable is independent of $\hat{\beta }$; it is also independent of the new error ${\epsilon }^{\ast }$. Therefore it is independent of the standardized prediction error, and the defining normal-over-chi-square representation of Student's t-distribution givesIf $t_{n-p,1-\alpha /2}$ is the corresponding upper quantile, symmetry yieldsSolving these inequalities for $Y^{\ast }$ proves that the prediction interval in a normal linear model isIts center and endpoints depend only on the observed data and $x^{\ast }$, and the pivotal calculation proves the required coverage uniformly in the unknown $\beta$ and ${\sigma }^2$.

For the normal linear model, the Gaussian likelihood givesMaximization over $\beta$ gives the ordinary least squares estimator $\hat{\beta }$, and maximization over the variance givesAt this value the quadratic term in $-2\ell$ is $n$. There are $p$ regression coefficients and one variance parameter, so substitution into the Akaike information criterion yields

For the normal linear model, the maximum-likelihood estimators areandThe divisor is $n$ for maximum likelihood, rather than the degrees-of-freedom divisor used by the unbiased residual-variance estimator.

Writing $Y_i={\mathrm{medv}}_i$, $L_i={\mathrm{lstat}}_i$, and $A_i={\mathrm{age}}_i$, the fitted normal linear model isThe R formula lstat * age includes both main effects and their interaction term.

Three notable features are:

Let $T_i>0$ be record time and let $x_i=(1,c_i,d_i{)}^T$ contain standardized climb and distance. Model 1 is the normal linear modelModel 2 applies the same model after a logarithmic transformation:so $T_i$ is conditionally log-normal. Model 3 is a Gamma regression with logarithmic link:

The other quantitative columns are mpg, psngr, length, width, and weight, so use

This is a normal linear model with an intercept and five slopes.

The fitted normal linear model isfor $i=1,\dots ,173$.

The normal linear model is $Y=X\beta +\epsilon$ with $\epsilon \sim N(0,{\sigma }^2{I}_n)$. Full column rank givesWith hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$, the $i$th regression leverage is $h_{ii}$. Since $e=(I-H)Y=(I-H)\epsilon$ and $I-H$ is an orthogonal projection,High-leverage observations can move their own fitted values strongly and may exert disproportionate influence; residual size alone can therefore conceal them.

Suppose independent future log responses satisfy $Z_j=x_j^T\beta +{\epsilon }_j$, where the errors have variance ${\sigma }^2$. Conditional on a fitted normal linear model, the predicted log ratio $Z_1-Z_2$ has estimated mean $(x_1-x_2{)}^T\hat{\beta }$ and estimated varianceA Student t interval on this logarithmic scale can be exponentiated to obtain a prediction interval for the positive ratio.

For the full-rank normal linear model $Y=X\beta +\epsilon$ and an independent future response $Y^{\ast }=x^{\ast T}\beta +{\epsilon }^{\ast }$, putThen an exact $(1-\alpha )$ prediction interval isThe additional one inside the square root is the future observation's irreducible error variance; a confidence interval for its mean omits it.

Replacing a numerical predictor $x$ by $x-c$ preserves the fitted mean space and slope in a linear regression containing an intercept. The intercept becomes the fitted mean at $x=c$. In simple linear regression, centering at the sample mean makes intercept and slope estimates uncorrelated under a normal linear model. This changes coefficient interpretation, not predictions.

In a fixed full-rank normal linear model, the studentized estimator of one coefficient has a noncentral t-distribution with noncentrality equal to that coefficient divided by its standard deviation. The fitted and residual orthogonal projections of the Gaussian noise are independent. Therefore the statistical power of its exact Student's t-test does not depend on the other regression coefficients, with design and variance fixed.

For a fixed residual direction $R=Z-X{\tilde{\beta }}_{\gamma }\ne 0$ obtained by Square-root Lasso, its optimality condition gives $\Vert X^{T}R{\Vert }_{\infty }\le \sqrt{n}\gamma \Vert R{\Vert }_2$. In a fixed-design normal linear model, the score $R^T(Y-X\hat{\beta })/\Vert R{\Vert }_2$ equals a $N(0,{\sigma }^2)$ component plus a remainder bounded by $\sqrt{n}\gamma \Vert {\beta }^0-\hat{\beta }{\Vert }_1$. The direction must be fixed or independent of the response noise for that normality claim.

In a full-rank normal linear model with $n$ observations and $p$ mean coefficients, the residual standard error estimates the common error standard deviation. Its square is the unbiased estimator $\mathrm{RSS}/(n-p)$; the square root itself is not generally unbiased. It measures unexplained response variation, whereas a regression coefficient's standard error measures uncertainty in that coefficient.

In a normal linear model, the $i$th regression residual has variance ${\sigma }^2(1-h_{ii})$, where $h_{ii}$ is a diagonal entry of the hat matrix. Replacing $\sigma$ by the residual standard deviation $s$ gives the standardized residual used in the normal Q-Q plot of R's linear-model diagnostic plots.

The number of free dimensions remaining after specified constraints often determines a reference sampling distribution. In a full-rank normal linear model with $n$ observations and $p$ mean coefficients, the residual space has dimension $n-p$. A regular nested likelihood-ratio test instead uses the difference in free statistical parameter dimensions. Boundary constraints may invalidate this regular calibration.

A two-factor normal linear model describes independent responses with factor-specific means and common normal error variance. The additive mean $\mu +{\alpha }_a+{\gamma }_j$ assumes no interaction term; the full mean $\mu +{\alpha }_a+{\gamma }_j+{\delta }_{aj}$ permits the first factor effect to vary with the second. Constraints or treatment coding identify the parameters. In a balanced two-by-five design the full model has ten free cell means.