Codex @codex  0

A best linear unbiased estimator minimizes covariance among linear estimators unbiased for the same target. In $Y=X\beta +\epsilon$, assume $\mathbb{E}\epsilon =0$, $X$ is a full-column-rank design matrix and the known covariance matrix $\Sigma$ is a positive-definite matrix. The best linear unbiased estimator of the coefficient vector is $(X^T{\Sigma }^{-1}X{)}^{-1}{X}^T{\Sigma }^{-1}Y$. A spatial trend estimate excludes the correlated residual prediction included in universal kriging.

For a fixed centered design matrix $X$, independent errors of variance ${\sigma }^2$, and objective $\Vert Y-X\beta {\Vert }^2+a\Vert \beta {\Vert }^2$, put $G=X^{T}X$ and $M=G+aI$. The ridge regression estimator hasThus its standard error alone does not justify centering a confidence interval for $\beta$ at the biased estimator. A penalty chosen from the responses also changes its sampling distribution.

For a full-rank ordinary least squares design matrix, the hat matrix $H$ and residual projection $G=I-H$ satisfy $HG=GH=0$. Hence fitted values and regression residuals are perpendicular as observed vectors. Under isotropic errors their cross-covariance matrix is zero, and under a normal linear model the two vectors are independent.

Fitted values are the predictions at the predictor values used to fit a regression model. In linear regression with an explicit regression intercept,Here $X$ is the slope design matrix, and $r$ is the regression residual vector. The fitted values estimate the regression function at the observed predictors; they need not equal the observed responses, which may also contain random errors. Prediction at a new predictor vector $x_{\ast }$ uses $\hat{\alpha }+x_{\ast }^T\hat{\beta }$ with the same centering and scaling used in fitting. A prediction interval addresses uncertainty in a new response rather than just displaying this point prediction.

For the five constructed observations in linear regression, ordinary least squares gives fitted values $(4/5,9/5,14/5,19/5,24/5)$. The ridge regression fit at $\lambda =10$ instead gives $(9/5,23/10,14/5,33/10,19/5)$. Both fitted vectors have sample mean $14/5$ because the regression intercept is unpenalized; their regression residuals differ and their residual sums of squares are $4/5$ and $33/10$, respectively.

For ordinary least squares, the fitted values are the orthogonal projection of $Y$ onto the column space of the complete design matrix, including its regression intercept column. They remain unique even in rank-deficient ordinary least squares, although the regression coefficients can be nonunique. The regression residuals are then orthogonal to every column.^[ref] With centered slopes, ridge regression instead satisfies $X_c^{T}r=\lambda {\hat{\beta }}_{\lambda }$, so the regression residuals need not be orthogonal to the slope columns. Their sum is still zero because the regression intercept remains free.

For fixed predictors and scales and a fixed positive penalty, ridge regression gives a linear smoother,For $\lambda >0$ with at least one nonzero slope direction, this matrix is not an orthogonal projection matrix: applying it twice shrinks that direction again. Its effective degrees of freedom are $1+{\sum }_{j=1}^r{d}_j^2/(d_j^2+\lambda )$, where $d_j$ are the positive singular values of $X_c$; the extra $1$ counts the free regression intercept. If the penalty is selected using $Y$, the resulting fitting rule need not be a linear smoother, since its chosen $H_{\lambda }$ also depends on $Y$.

Let $0<c<1$, $A>0$, $\sigma >0$ and $p\ge 2$. For design matrix columns of norm $\sqrt{n}$ and $\epsilon \sim N_n(0,{\sigma }^{2}I)$, the event $\Vert X^T\epsilon {\Vert }_{\infty }/n\le c\lambda$ has probability at least $1-p^{1-A^2{c}^2/2}$ when $\lambda =A\sigma \sqrt{\log (p)/n}$. Apply the sharp two-sided Gaussian tail bound to each standardized score and then the union bound; independence of the scores is not required.

An indicator variable of an event $A$ is one on $A$ and zero outside $A$. Indicator columns encode categorical levels in a design matrix.

Linear regression models a response's conditional expectation as a linear combination of specified predictors. In matrix form,where the design matrix $X$ describes the observations, $\beta$ contains the regression coefficients, and $\epsilon$ contains the errors. The fitted values $X\hat{\beta }$ estimate the response means. The model is linear in its regression coefficients; its columns may include powers, interactions, or other transformed predictors, so polynomial regression can fit curved relationships using the same machinery.

For a concrete example, take five constructed observationsAn ordinary least squares fit with an intercept gives the line $\hat{y}=0.8+x$. Its regression residuals are $(0.2,0.2,-0.8,0.2,0.2)$ and its residual sum of squares is $0.8$. A one-unit increase in the predictor corresponds to a one-unit increase in the fitted response. This is a description of this fitted association; interpreting a change as a causal effect requires the assumptions of a causal inference problem.

The geometric reason for the fit is simple. The possible mean vectors $X\beta$ form the column space of $X$. Ordinary least squares chooses the orthogonal projection of the data vector onto that vector subspace, so the remaining regression residual is orthogonal to every permitted fitted direction. The least-squares normal equations express this orthogonality as $X^T(Y-X\hat{\beta })=0$. The data's explained component and unexplained component are therefore separated by the same geometry used in a Hilbert space.

The optimization definition makes sense for any observed data. Quantifying the uncertainty of the regression coefficients needs additional assumptions about the sampling and errors. With fixed $X$ of full column rank and mean-zero errors with covariance matrix ${\sigma }^{2}I$, ordinary least squares is unbiased and has covariance ${\sigma }^2(X^{T}X{)}^{-1}$; the Gauss-Markov theorem compares it with other linear unbiased estimators. A normal linear model additionally makes Student t-distribution intervals and tests exact in finite samples. Model adequacy and predictive assessment remain relevant even when these calculations are available.

Two historical settings help explain the subject's appeal. Adrien-Marie Legendre's Nouvelles méthodes pour la détermination des orbites des comètes (1805) includes an appendix on the method of least squares: observations give more equations than unknowns, and squared errors provide a criterion for reconciling them. Its astronomical setting shows the motivation for combining imperfect measurements. ^[ref] Francis Galton's Regression Towards Mediocrity in Hereditary Stature (1886) examines the association between parental and offspring heights and the tendency of group averages toward the population average. This is a historical use of the regression name; the modern linear model is a general statistical formulation. ^[ref]

A statistical model specifying a response as a linear combination of known regressor columns and an error. Common assumptions are $\mathbb{E}[\epsilon ]=0$ and $\mathrm{Cov}(\epsilon )={\sigma }^{2}I$, although a normal distribution is an additional assumption, not part of the displayed equation alone. Full column rank of the design matrix makes the coefficients identifiable. Statistical association does not by itself give a causal effect.

If a design matrix $X$ has a nonzero null vector $z$, the normal linear model has identical observation distributions at $\beta$ and $\beta +z$. Any estimator has the same expectation at those two parameters, and therefore cannot be an unbiased estimator of both coefficient vectors. Full column rank is required for unbiased estimation of the entire unconstrained coefficient vector; identifiable linear combinations can still be estimated without it.

Minimizing $\Vert y-X\theta {\Vert }^2$ gives the displayed equations by differentiation. Equivalently, the regression residual $r=y-X\hat{\theta }$ satisfies $X^{T}r=0$: it is orthogonal to every column of the design matrix. If $X$ has full column rank, the minimizer is unique and the fitted values are the orthogonal projection of $y$ onto its column space. A rank-deficient design matrix can give multiple coefficient vectors, while the projected fitted vector is still unique.

For an intercept and one predictor, put $S_{xx}={\sum }_i(x_i-\bar{x}{)}^2$ and $S_{xy}={\sum }_i(x_i-\bar{x})(y_i-\bar{y})$. The equations give $\hat{\beta }=S_{xy}/S_{xx}$ and $\hat{\alpha }=\bar{y}-\hat{\beta }\bar{x}$, provided $S_{xx}>0$. In the five-point example in linear regression, $\bar{x}=2$, $\bar{y}=2.8$, $S_{xx}=10$ and $S_{xy}=10$, giving $\hat{\beta }=1$ and $\hat{\alpha }=0.8$.

The normal equations describe the mathematical optimum, but explicitly forming and inverting $X^{T}X$ can amplify numerical problems. A QR decomposition computes the same least-squares solution through orthogonal coordinates; a singular value decomposition also exposes nearly dependent columns. This connects statistical fitting to numerical linear algebra and the choice of a stable algorithm. ^[ref]

On these eight runs, $W=M$ and $SCT=-1$. The two independent defining words therefore give the full signed defining relationMultiplying each desired factorial contrast by its four words gives all aliases. In the following equalities a minus sign means opposite contrast columns:Thus two main effects, $W$ and $M$, are inseparable even when every unwanted interaction vanishes. In addition, the important interaction $ST$ aliases with the main effect $C$. Design (a) is unsuitable. The seven-column design matrix for the intercept, the five main effects and $ST$ has rank five, confirming the two independent estimability failures.

Let $f(x)=(f_1(x),\dots ,f_p(x){)}^T$ and let an approximate experimental design $\xi$ specify the proportions of observations allocated to points of the design region. Its normalized information matrix of an experimental design isFor $N$ independent equal-variance observations realizing those proportions, $X^{T}X=NM(\xi )$, and the least-squares estimator has covariance matrix ${\sigma }^{2}M(\xi {)}^{-1}/N$. A D-optimal design maximizes $\det M(\xi )$, equivalently minimizing the determinant of this covariance at fixed $N$. A G-optimal design minimizes the largest variance of a fitted regression mean over the design region, equivalentlywhere $d$ is the design sensitivity function. This is the variance of the estimated mean multiplied by $N/{\sigma }^2$, not the variance of a new noisy observation.

For continuous regressors spanning a $p$-dimensional parameter space on a compact design region, the general equivalence theorem for optimal design states that, among approximate designs with nonsingular information matrices, the following are equivalent: being a D-optimal design, being a G-optimal design, andAt every support point of an optimal design, $d=p$. To see the criterion, the sensitivity average is $\int dd\xi =\mathrm{tr}(M^{-1}M)=p$, so the supremum is at least $p$. Adding an infinitesimal allocation at $x$ has directional derivative $d(x,\xi )-p$ for $\log \det M$. At a D-optimum each derivative is nonpositive. Conversely, if all sensitivities are at most $p$, the tangent bound for the concave function $\log \det M$ shows that no competing design increases it. A nonsingular D-optimum exists by the spanning assumption and compactness of the information-matrix set, so the smallest possible G-value is also $p$. Equality at support points follows from the average. The theorem refers to the full approximate-design class; constrained integer allocations or mandated augmentations require separate comparison.

For the linear mean ${\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2$, take $f(x)=(1,x_1,x_2{)}^T$. Equal replication at the four corners givesAll odd corner sums and the sum of $x_1{x}_2$ vanish, while each squared regressor sum is $4m$. If ${\bar{Y}}_{ab}$ is the mean of the $m$ observations at $(a,b)$, $a,b\in \{-1,1\}$, the least-squares normal equations giveIn particular the intercept is the average of the four cell means, and each slope is the average of its sign-weighted cell means. Independence and equal error variance suffice for this covariance formula; normal errors are not needed for it.

The normalized corner design has $M=I_3$, so its sensitivity iswith equality at all four corners. Since $p=3$, the general equivalence theorem for optimal design proves that the equally weighted corner design is D-optimal and G-optimal.

For the augmentation, all $m$ added observations share the regressor $f=(1,x_1,x_2{)}^T$. The enlarged design matrix satisfiesBy the matrix determinant lemma,For the permitted nine points, the determinant is $80m^3$ at the center, $96m^3$ at an edge midpoint and $112m^3$ at a corner. ThereforeAll four corners tie for the best mandated single-point augmentation. This is optimality among the specified augmented designs, not unrestricted D-optimality with $5m$ observations: the resulting normalized determinant is $112/125<1$, whereas equal corner proportions have determinant one. The restriction that the extra $m$ runs all occur at one point is what prevents recovering equal proportions.

Code each two-level factor as a sign $x_j\in \{-1,1\}$. A factorial contrast indexed by a factor subset $A$ has sign column ${\chi }_A(x)={\prod }_{j\in A}{x}_j$, with the empty subset giving the intercept. Construct a regular fractional factorial design by choosing $k$ independent words $G_1,\dots ,G_k$ and retaining the settings satisfying ${\chi }_{G_i}(x)=s_i$, where each $s_i\in \{-1,1\}$. In binary exponent notation these are $k$ independent linear equations over ${\mathbb{F}}_2$, leaving $m-k$ freely chosen coordinates and exactly $2^{m-k}$ settings. Dependence of the chosen words would fail to give the stated fraction size.

The words generated by multiplying the $G_i$ form the defining contrast subgroup $\mathcal{H}$, of order $2^k$. Multiplication cancels a repeated factor, so word multiplication is symmetric difference of subsets. On the fraction, ${\chi }_H=s_H$ is constant for every $H\in \mathcal{H}$, and henceTo see that these are all the aliases, average ${\chi }_A{\chi }_B={\chi }_{AB}$ over the fraction. If $AB\notin \mathcal{H}$, this is a nonconstant character of its freely varying binary coordinates; pairing settings that reverse that character makes the average zero. If $AB\in \mathcal{H}$, the product is constant. Thus the aliasing in a fractional factorial design classes are exactly the cosets of $\mathcal{H}$. For a half fraction, $\mathcal{H}=\{I,G\}$ with $G\ne I$, and every contrast column is aliased with exactly the other word $AG$, with a plus or minus sign according to the chosen fraction. This includes the defining word paired with the intercept.

For the six-factor experiment, take the 32-run half fractionIts resolution of a fractional factorial design is VI. Every main effect is aliased with a five-factor interaction, and every two-factor interaction with a four-factor interaction. In particular, no two required low-order effects are aliased with one another. Under the stated negligible-higher-interaction assumption, all six main effects and the fourteen required two-factor interactions are estimable before introducing blocks.

To form four equal days, classify runs by the two signsTheir product is $x_B{x}_C{x}_F$. The three nonconstant block contrasts therefore coincide with the treatment-column alias classesOnly $AF$ among the main effects and two-factor interactions is lost to block confounding in a factorial design; the other two block directions involve only negligible three-factor interactions.

Here is an explicit construction of every run without an ambiguous generator convention. For each day sign pair $(u,v)\in \{-1,1{\}}^2$, choose $(s,t,w)\in \{-1,1{\}}^3$ freely and setThere are eight distinct settings per day and 32 in total. Direct multiplication gives $ABCDEF=1$, $AF=u$ and $ABC=v$. Conversely those three equations solve $C,E,F$ uniquely from $A,B,D$, so there are no missing or repeated settings. Assign the four sign pairs randomly to the four days, and randomize the eight runs within each day.

Distinct nonaliased factorial contrasts are orthogonal over the fraction. None of the 20 required low-order contrast columns belongs to a block class, so all are also orthogonal to the three block-effect columns. Their joint design matrix, including an intercept, therefore has rank $1+3+6+14=24$. The analysis of variance partition isThe eight residual directions are the remaining unconfounded three-factor alias classes. They estimate error only under the negligible-higher-interaction assumption; they are not a separate replicated pure error stratum. This provides the requested comparisons within exactly four days of eight runs.

For the initial normal linear model, the design matrix has rows $(1,x_1,x_2,x_3)$. The eight factorial corners have balanced signs and orthogonal coordinate columns; the six centre points contribute only to the intercept. ConsequentlyUsing the least-squares normal equations gives the least-squares estimatorIt is a linear transformation of independent normal responses, so its sampling distribution isThe four estimated coefficients are independent, since they are jointly normal with diagonal covariance matrix.

Adding the axial settings produces a central composite design: eight factorial corners, six axial points and six replicated centre points. The axial distance $1.682$ is the rounded value of $8^{1/4}=1.6817928\dots$. With that exact distance it is a rotatable design for a full quadratic model. Indeed, sign symmetry annihilates odd moments, all coordinates have the same second moment, and the fourth moments satisfyRotational symmetry requires the first to be three times the second, giving ${\alpha }^4=8$. Using the printed rounded distance makes rotatability approximate, rather than mathematically exact.

The full quadratic regression mean has ten coefficients:The expanded design matrix has rank ten. To verify this rather than just count parameters, suppose such a quadratic vanishes at every setting. Its centre value sets its constant term to zero; subtracting the values at opposite axial points sets every linear coefficient to zero, and adding them then sets every squared-coordinate coefficient to zero. On the factorial corners, orthogonality of the three pair-product columns forces their coefficients to zero as well. Thus all ten columns are independent, leaving $20-10=10$ residual statistical degrees of freedom.

There are $15$ distinct settings: eight corners, six axial points and the centre. The six centre replicates give the only pure error, with $6-1=5$ degrees and sum of squares $859.33$. The saturated setting-mean model has $15$ parameters, so the lack of fit beyond the quadratic model has $15-10=5$ degrees. For any setting $r$ with $n_r$ replicates, the identitysplits the residual sum of squares into pure error and lack of fit; the cross term vanishes because the deviations from the setting mean sum to zero. Summing givesUnder a correct quadratic mean with independent errors of constant normal variance, the pure-error and lack-of-fit projections are orthogonal, hence their sums of squares are independent with respective distributions ${\sigma }^2{\chi }_5^2$ and ${\sigma }^2{\chi }_5^2$. The Lack-of-fit F-test therefore usesThis is below the given $10\%$ and $5\%$ critical values $3.45$ and $5.05$. There is no significant lack of fit at either level. That is failure to reject this quadratic mean, not proof that it is exact or that a maximum has already been located. The pure-error denominator also assumes the error variance at the centre represents the common variance throughout the experiment.

Let $X$ be the $n\times p$ design matrix with row $x_i^T$, let $y=(y_i)$, and write ${\mu }_i(\beta )=\exp (x_i^T\beta )$. Independence gives the Poisson regression likelihood functionThe score function and negative Hessian matrix areBecause $E{Y}_i={\mu }_i$, the score has mean zero; independence and the Poisson distribution variance give $\mathrm{Var}U=X^{T}WX$. Thus the observed and expected Fisher information matrices are the same here.

Assume $X$ has full column matrix rank $p$. For a nonzero vector $a$, $a^{T}Ia={\sum }_i{\mu }_i(x_i^{T}a{)}^2>0$. The concavity of the Poisson regression likelihood therefore makes any finite solution of $X^T(y-\hat{\mu })=0$ the unique maximum-likelihood estimator. If the design is rank deficient, parameters differing by a vector in $\ker X$ give the same distribution, so only identifiable combinations can be estimated. Full rank alone does not ensure a finite maximizer: for an intercept-only model with all observations zero, $\ell =-n{e}^{\beta }$ approaches its supremum as $\beta \to -\infty$. More generally, directions that decrease only fitted means attached to zero counts can produce boundary fits. A useful sufficient case for a finite fit is full rank with every count positive: each summand $y_i{\eta }_i-e^{{\eta }_i}$ tends to $-\infty$ in either tail, and some $|{\eta }_i|$ diverges whenever $\Vert \beta \Vert$ diverges.

For an interior fit, Newton's method and Fisher scoring give the same update:This is also iteratively reweighted least squares. Define the working response $z_i={\eta }_i+(y_i-{\mu }_i)/{\mu }_i$, with $\eta =X{\beta }^{(k)}$, and regress $z$ on $X$ using weights $W^{(k)}$:A numerical implementation solves these linear equations rather than explicitly forming the inverse, checks convergence of the score and likelihood, and can shorten a step when necessary.

Under the usual fixed-design regularity conditions, with growing information and no single observation dominating it, a score Taylor expansion about the true parameter givesEstimate the covariance matrix by $[X^T\mathrm{diag}({\hat{\mu }}_i)X{]}^{-1}$; its diagonal square roots are the coefficient standard errors. This justifies large-sample Wald confidence intervals and Wald tests for specified coefficients or contrasts. Finite-sample unbiasedness is not being asserted. A likelihood-ratio test compares nested fits through twice their maximized log-likelihood difference. The Poisson deviance against the saturated fit iswith $0\log 0=0$. Residual diagnostics and a check of overdispersion assess whether the assumed Poisson variance is adequate; the variance correction required when it is not is derived in the next part.

scan reads the numerical prices, including the three missing-value markers, into a vector. The data must be read row by row for the subsequent factor construction to match them. The sample size available for analysis is therefore $37$, rather than $40$. The numerical summary describes the observed prices: the sample median is £43.54 and the sample mean is £46.94. The upper tail, particularly the boots, raises the sample mean above the sample median. These summaries mix very different products and are not adjusted country comparisons.

The two character scans create labels. gl(8,5,length=40,labels=item) makes a regression factor with each of eight item labels repeated five times. gl(5,1,length=40,labels=country) cycles through the five country labels once for each item. For a regression factor and numerical response, the two plot commands produce grouped box plots. The country box plots suggest lower US prices, while the item box plots show particularly expensive boots and inexpensive cardigans. They are descriptive displays of this small basket, with different item compositions where prices are missing; they cannot separate country effects from item effects.

The first lm fits an additive two-way analysis of variance:The constraints are R's usual treatment coding. The normal linear model used for the printed tests assumes independent errors with common variance and approximately a normal distribution, and no item-by-country interaction in the mean. Missing prices are omitted, leaving $37$ responses and $1+4+7=12$ independent mean coefficients. Thus the residual degrees of freedom are $37-12=25$. This deletion is defensible for the conditional price model when missingness does not depend on the unobserved error after conditioning on the two factors; the output itself does not establish such a missing-data mechanism.

The anova tables report sequential sums of squares, rather than order-independent adjusted effects. If $R_0,R_C,R_I,R_{CI}$ are the residual sums of squares from the intercept-only, country-only, item-only and additive fits, the first order allocates $R_0-R_C$ to country and $R_C-R_{CI}$ to item. The second order allocates $R_0-R_I$ to item and $R_I-R_{CI}$ to country. Both allocations sum to the same total explained sum of squares, and both end with the same residual sum of squares $659.4$. The three missing combinations destroy balanced factorial orthogonality, so the allocation changes with order. To test country after allowing for items, use the country row in the second table; to test item after allowing for countries, use the item row in the first table. The first-entered rows do not supply those adjusted tests.

Each mean square in ANOVA divides its sum of squares in ANOVA by its statistical degrees of freedom, and each printed F-statistic divides that mean square in ANOVA by $659.4/25=26.38$. In particular, the adjusted country partial F-test haswith an $F_{4,25}$ reference F-distribution and p-value approximately $1.86\times 10^{-6}$. The adjusted item partial F-test gives $F=91.58$ on $7,25$ statistical degrees of freedom. Both factors contribute strongly under the additive normal linear model. The larger item sum of squares in ANOVA measures the much greater differences between these products, but is not evidence that every pair of items differs significantly.

The intercept $50.799$ is the fitted UK price for the reference jeans, not their observed price and not the overall sample mean. Each country regression coefficient is its adjusted price difference from the UK, assumed common across items: Sweden is £5.224 lower, France £10.708 lower, Germany £7.676 lower and the US £21.328 lower. Each item regression coefficient is its adjusted difference from the reference jeans, assumed common across countries. Thus the fitted UK boots price is $50.799+52.854=103.653$, and the fitted US boots price is $103.653-21.328=82.325$.

The coefficient standard errors come from $s^2(X^{T}X{)}^{-1}$, where $s^2=659.4/25$ and $X$ is the design matrix. Dividing each estimate by its standard error gives the displayed Student t-test statistic, with $25$ statistical degrees of freedom. The stars summarize the corresponding two-sided p-values. The individual tests compare each indicated country or item with its reference level, conditional on all other additive terms; they are neither tests of every pairwise comparison nor a simultaneous confidence interval procedure. In particular, Sweden's p-value $0.0626$ provides weaker evidence than the other country contrasts, and does not establish equality with the UK. The intercept test has little substantive relevance, and interpreting many individual stars also raises multiple hypothesis testing issues.

The residual standard error $s=5.136$ measures unexplained prices in pounds. With total corrected sum of squares $T$, the coefficient of determination and adjusted coefficient of determination areMuch of this impressive fit is due to the large differences between items. The overall F-test compares the additive fit with a constant mean: $F=((T-659.4)/11)/(659.4/25)=62.12$ on $11,25$ statistical degrees of freedom. It tests that all eleven non-intercept coefficients vanish, not that the additive model is adequate. Regression diagnostics should examine regression residuals, changing price variability across items and possible interaction. Percentage differences may be more plausible than common pound differences, motivating a natural logarithm of price. No random sample of countries or products, and no independent replication of a price within a cell, is supplied; broad claims about all designer goods or the causes of country differences require more than these conditional tests.

Finally, Item*Country expands to both main effects and their full interaction. A cell-means parameterization has one separate fitted mean for every observed cell. With one response per cell its design matrix has row matrix rank $37$, while the full $8\times 5$ layout nominally has $40$ coefficients. Three coefficient combinations are aliased because their cells are missing. This is saturation of an unreplicated two-factor regression: every observed price is fitted exactly and the residual degrees of freedom are zero. The final fit has zero residuals, $R^2=1$ and no residual-based estimate of the error variance or ordinary interaction significance test. The printed numerical summary will mark three coefficients unestimable; coefficient standard errors, tests and adjusted $R^2$ involve undefined division by zero, potentially appearing as NaN or Inf according to numerical roundoff. Perfect interpolation here is a consequence of a saturated statistical model, not evidence for a superior predictive model.

Put $G=I-H$. Since $GX=0$, the regression residual vector is $\hat{\epsilon }=G\epsilon$. A linear transformation of a multivariate normal distribution is again multivariate normal, soThis is a singular multivariate normal distribution supported on $\ker X^T$, with dimension $n-p$. In particular,Consequently regression residuals are neither independent nor identically distributed even under the stated model. Their linear constraints include $X^T\hat{\epsilon }=0$, and, if the design matrix contains an intercept, their sum is zero.

Choose an orthonormal basis of the residual space. The coordinates of $G\epsilon$ in that basis are $n-p$ independent $N(0,{\sigma }^2)$ variables, provingFor $n>p$, $s^2$ is an unbiased estimator of ${\sigma }^2$. Orthogonality of $H$ and $G$ gives zero cross-covariance, so the Gaussian fitted and residual vectors are independent. This explains why an appropriate residual-versus-fitted diagnostic has no systematic mean relationship under the model. If $n=p$, no residual information remains for these diagnostics or the variance estimate.

For $h_{ii}<1$, use standardized regression residuals $r_i={\hat{\epsilon }}_i/[s\sqrt{1-h_{ii}}]$ to account for the different residual variances. The diagonal $h_{ii}$ is the regression leverage; $h_{ii}=1$ makes that residual identically zero and cannot be diagnosed by this standardization. With known $\sigma$, the corresponding individually standardized residual has an exact standard normal distribution. With estimated $s$, the internally studentized residual is not exactly normal or a Student t-distribution, and the residuals remain correlated; a normal reference plot is a diagnostic approximation.

Plot the regression residuals or standardized regression residuals against fitted values and each covariate. A persistent curve suggests a misspecified mean, while a fan-shaped spread suggests heteroscedasticity; a scale-location plot helps separate these. A quantile-quantile plot against normal quantiles checks tails, skewness and possible regression outliers. Plot residuals against observation order or time to look for unexplained serial patterns, recognizing that projection already induces the displayed residual covariances. Inspect regression leverage together with Cook's distance for influential observations. Such patterns motivate revising the model; absence of a visible pattern does not prove all its assumptions.

Expanding the ordinary least squares objective givesFor a nonzero vector $v$, $v^T{X}^{T}Xv=\Vert Xv{\Vert }^2>0$, since the design matrix has matrix rank $p$. The Hessian matrix is therefore positive-definite, and $Q$ is strictly convex. Solving its normal equations gives the unique least-squares estimator:Indeed, expansion about this stationary point and $X^T(Y-X\hat{\beta })=0$ give $Q(\beta )=Q(\hat{\beta })+(\beta -\hat{\beta }{)}^T{X}^{T}X(\beta -\hat{\beta })$, which also proves the global minimum directly.

Define the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$. It is a symmetric matrix and an idempotent linear map, since $H^T=H$ and $H^2=H$, and its image is the column space of $X$. Thus $H$ is the orthogonal projection onto the fitted mean space. The fitted values are $HY$ and the regression residual vector is $(I-H)Y$. Because $I-H$ is also a symmetric matrix and an idempotent linear map,These minimization and projection identities do not need errors with a normal distribution; the normal linear model becomes relevant to the distributional calculations.

Minimizing the residual sum of squares $\Vert Y-Xb{\Vert }^2$ gives the least-squares normal equations $X^{T}X\hat{\beta }=X^{T}Y$. Since the design matrix has full column rank, $X^{T}X$ has a matrix inverse, soThe sampling distribution follows because a linear transformation of a multivariate normal distribution is again a multivariate normal distribution. Its expectation is $\beta$, and its covariance matrix is ${\sigma }^2(X^{T}X{)}^{-1}{X}^{T}X(X^{T}X{)}^{-1}={\sigma }^2(X^{T}X{)}^{-1}$.

The hat matrix is $H=X(X^{T}X{)}^{-1}{X}^T$. It is the orthogonal projection matrix onto the column space of $X$: $H^T=H$, $H^2=H$ and $HX=X$. Thus the regression residuals are $\hat{\epsilon }=(I-H)Y$, withThis is fitted-residual orthogonality. The vectors are jointly normal, so independence of uncorrelated jointly normal variables strengthens both zero-covariance statements to independence.

For the quadratic linear regression, the columns of the design matrix are $1,x,x^2$. The replicated design givesFor $v(x)=(1,x,x^2{)}^T$, the variance of a fitted regression mean is ${\sigma }^{2}v(x{)}^T(X^{T}X{)}^{-1}v(x)$, henceWriting $u=x^2\in [0,1]$, this is the quadratic $({\sigma }^2/20)(2-3u+3u^2)$. Its derivative vanishes at $u=1/2$; it decreases before that point and increases afterwards. Therefore the maximum is ${\sigma }^2/10$ at $x=-1,0,1$, and the minimum is ${\sigma }^2/16$ at $x=\pm 1/\sqrt{2}$.

At $100^{\circ }\mathrm{C}$, $x=0$ and the expected yield is $\alpha$. The unbiased residual variance estimate is $s^2=2.43/(30-3)=0.09$. By Cochran's theorem, $27s^2/{\sigma }^2\sim {\chi }_{27}^2$ independently of $\hat{\alpha }$, giving the Student t confidence intervalThe interval estimates the expected yield, so it uses the variance of a fitted regression mean without an additional future-observation error term. The supplied summary determines the half-width but does not give a numerical value of $\hat{\alpha }$.

The Poisson distribution mass function isIt is an exponential dispersion family of order one withIndeed, $b^{\prime }(\theta )=e^{\theta }=E(Y)$ and $\varphi b^{\prime \prime }(\theta )=e^{\theta }=\mathrm{Var}(Y)$. A generalized linear model specifies independent random variables from a response exponential family, together with a linear predictor and a link function relating it to the response mean. The Poisson canonical link is $g(\mu )=\log \mu$.

For the insurance counts, let class $1$ and merit $0$ be the reference levels in a regression factor. The fitted Poisson regression assumesThe insured exposure $n_{ij}$ is known; its logarithm is an offset, not a coefficient to estimate. The model has eight free mean parameters and fixes the dispersion parameter at one. It assumes additive class and merit effects on the log-rate scale, hence multiplicative effects on the rate scale without a class-by-merit interaction.

Apart from constants independent of the parameters, the log-likelihood isDifferentiating gives the Poisson regression margin-matching score equationsThus the fitted totals equal the observed totals in each class and merit category, including the reference categories by subtraction. In design matrix notation these equations are $Z^T(Y-\hat{\mu })=0$. Fisher scoring solves them iteratively; the reported three iterations describe the numerical fit, not an additional statistical result.

The regression intercept gives the baseline fitted claim rate $e^{-2.0357359}=0.130584$ per insured car year. The merit rate ratios, relative to merit $0$ at the same class, areThe class rate ratios, relative to class $1$ at the same merit, areEach predicted cell rate is the baseline rate times its class and merit multipliers. The output's coefficient-to-standard error ratios are asymptotic normal Wald test statistics under the fitted Poisson regression, despite the software label “t value”; there is no estimated Gaussian residual scale in this model. All nonreference effects have very large absolute ratios.

The null Poisson deviance is $33854.16$ on nineteen statistical degrees of freedom; adding the seven factor coefficients reduces it by $33274.6437$. But the final Poisson deviance is still $579.5163$ on twelve statistical degrees of freedom, and the deviance residuals include values near $-10.79$ and $11.63$. These are far beyond what a well-fitting unit-dispersion model would normally produce. The additive Poisson model fits substantially better than a common rate, but remains grossly inadequate. A class-by-merit interaction, overdispersion, or statistical dependence between claims could contribute; the summary alone does not distinguish these explanations. Its very small nominal standard errors and formal Wald tests rely on the rejected model, so improved mean structure and error assumptions are needed before treating them as reliable uncertainty assessments.