The intercept-only residual sum of squares contains both main-effect sums and the additive-model residual: . Its residual degrees of freedom are .
Let , and . The proposed least-squares estimators satisfy the sum-to-zero constraints because and . Define . Direct summation gives
Now write any other admissible parameters as , , , where . Expanding the ordinary least squares criterion, the cross-products with vanish by the preceding identities. The cross-products between , and vanish by their zero sums. Consequently
Every added term is nonnegative and all vanish only at , . This proves the unique constrained minimum, with
The additive-model residual sum of squares and intercept-only residual sum of squares are therefore
The same orthogonality gives the analysis of variance identity
Fitting factor B alone yields fitted values and residual sum . Fitting A alone gives residual sum . Adding B after A then reduces it to , a reduction of again. The reduction due to B is in either order. This is balanced factorial orthogonality; it generally fails for unequal cell replication.
There are observations. The additive model estimates independent parameters, so the missing degrees of freedom are A: 2, B: 1, residuals: 8. The individual requested residual sums are below.
To check for an interaction, extend the normal linear model to , with each row and column sum of zero. Equivalently, fit the six unrestricted cell means. Its residual sum of squares is the within-cell sum
The interaction adds parameters; the cell-means residual has degrees of freedom. Under the no-interaction null hypothesis and the independent equal-variance normal error model,
The numerator and denominator arise from orthogonal Gaussian projections, proving the exact F-test. Reject for a sufficiently large value. The printed additive analysis of variance table does not supply , so it cannot determine this test statistic numerically; the replicated cell data would do so.
Let be the flight distance in millimetres for paper level , launch-angle level , design level and replicate , where and . The full factorial design is fitted by the normal linear model
The assumptions are independent errors with a common variance and normal distribution, and the displayed factor-combination mean structure. The corner-point constraints are
and whenever at least one index equals . Thus the eight free mean coefficients are . These treatment coding coefficients are comparisons at the other factors' reference levels, rather than necessarily averaged main effects.
To test the three-factor interaction term, compare the full model with the model retaining all main effects and all two-factor interactions. The null hypothesis is , against a nonzero value. Removing it increases the residual sum of squares by and removes one parameter. With eight residual degrees of freedom, the nested-model F-test gives
There is no evidence that this three-factor interaction is needed. This does not establish that all two-factor interactions vanish: the paper-design interaction is substantial.
The stepAIC command performs stepwise selection by the Akaike information criterion. Its upper scope is the full factorial model and its lower scope is the intercept-only model. Starting from the full model, it compares admissible additions and deletions, preserving the hierarchy of interaction terms, and chooses changes that reduce the Akaike information criterion. With the supplied scope its default search permits both directions. For these Gaussian fits the displayed criterion, up to constants common to all models, is , where counts mean coefficients. The first deletion changes it from to : the small increase in residual sum of squares is outweighed by removing a parameter. This is a greedy model selection procedure, not an exhaustive search or a sequence of fixed-level hypothesis tests. The command's arguments are described in the MASS documentation.
For the selected model, let and . Then
independently between flights. It has no angle term. The maximum-likelihood estimates of its mean coefficients are also obtained by ordinary least squares:
For example, the four fitted means are just the four paper-design cell means, each averaging four flights over the angle levels. With , the variance-covariance matrix of the coefficient estimates is . Its diagonal gives the displayed standard errors; their ratios are , with mm.
The final line of summary(model2.lm) is the joint nested-model F-test of , comparing this four-parameter model with a common mean. Its statistic is
with F-distribution under the fixed-model null and P-value . Thus the selected paper-design terms jointly improve the fit substantially. The coefficient of determination means they account for about of the observed centred variation. These nominal inferential summaries do not adjust for the preceding model selection on the same data.
Using the unrounded fitted means gives
in millimetres. Hence choose the high-performance design with the lighter paper; the fitted model leaves both launch angles tied, with expected distance about mm. A preference for one angle is not supported by this selected model. The strong paper-design interaction reverses the design comparison: the simple design is better on heavy paper, whereas the high-performance design is better on light paper. Consequently a single design effect averaged over paper types would conceal the main result. There is little evidence here for an angle effect, and considerable flight-to-flight variability remains.
Before fitting, inspect raw-distance dot plots for all eight factor combinations, and interaction plots of cell means against design, with separate lines for paper and panels for angle. With only two replicates per cell, showing individual observations is more informative than elaborate box plots. These reveal reversals, spread and possible unusual flights. Afterwards inspect a residual-versus-fitted plot, a Q-Q plot of residuals, a scale-location plot, residuals against each factor and, if recorded, flight order, and Cook's distance for influential observations. Residual patterns by angle would question its deletion, while unequal spread or departures from the normal distribution would question the error assumptions rather than merely demand another mean term.
Put and . The rank of a matrix assumption makes invertible, and is the orthogonal projection matrix onto the column space of the design matrix. The Gaussian likelihood has log-likelihood
Minimizing the squared Euclidean norm gives the ordinary least squares normal equations . Maximizing over then gives the maximum-likelihood estimators
The divisor for the maximum-likelihood estimator of the variance is , whereas the unbiased residual variance estimate is . The exceptional event has probability zero under the Gaussian linear model with ; on that event the likelihood is unbounded as .
For the joint sampling distribution, write and . Their cross-covariance matrix vanishes because . These two vectors are jointly multivariate normal, so they are independent random variables. The residual orthogonal projection has rank , hence the standard chi-squared distribution result gives
Thus these two displayed marginal laws, with their independence, specify the full joint distribution.
The quadratic form
has a chi-squared distribution with degrees of freedom and is independent of . Their ratio therefore supplies the exact F-distribution pivot
If is the quantile of that F-distribution, inversion gives the normal linear-model confidence ellipsoid
For every and every , the probability that this confidence set contains is exactly . To test the null hypothesis, reject precisely when lies outside this confidence ellipsoid, equivalently when . Under the null hypothesis the rejection probability is exactly , independent of the unknown variance.
For the numerical calculation, the second column of the design matrix sums to zero and its sum of squares is . Direct calculation from the observations gives
Consequently the ordinary least squares estimate, residual sum of squares and unbiased residual variance are
The maximum-likelihood estimator of the variance is instead . At the proposed null value the F-distribution statistic is
Therefore reject the null hypothesis at the 5% level. The relevant critical value has numerator and residual degrees of freedom.
The reduced model combines the 0-, 3-, and 6-point categories. Relative to the full model, the null hypothesis is , leaving the 9-point contrast unrestricted. These are two linear restrictions on an otherwise unchanged normal linear model.
For nested normal models, the reduction in residual sum of squares divided by has a chi-squared distribution with degrees of freedom equal to the number of restrictions. It is independent of the full model's residual variance estimate. Hence the F-test statistic is
It is below the supplied 5% critical value , so do not reject equal effects for 0, 3, and 6 points. There is insufficient evidence that separating those categories improves this additive model; this does not prove that their true premiums are identical.
For the normal linear model, differentiating the ordinary least squares objective gives . Full column rank makes invertible, so
The distribution follows because is a linear transformation of a multivariate normal distribution. Its expectation is , and direct multiplication gives the displayed covariance matrix.
Let , the orthogonal projection onto the design's column space. The residual sum of squares is . Since , symmetry and idempotence give . Thus
is an unbiased estimator.
For the two independent regressions choose . The first rows of the design matrix are and the last rows are . The centering conditions and distinct predictors give
It follows that
The two estimated slopes are independent normal random variables, as they depend on independent error blocks. Therefore
For these data use the pooled residual sum of squares
Its scaled value is and is independent of the slope estimates by the permitted normal-model result. Hence the standardized difference has a Student t-distribution with degrees of freedom, giving the exact 95 percent confidence interval
Here is the 0.975 quantile of that distribution. This is a confidence interval for the difference of independent regression slopes. As required by the original framework, this specialization needs . Distinct predictors alone allow , but then the fit is saturated, and there are no residual degrees of freedom to estimate the unknown variance or form this interval.
Represent the subset model by a set of column indices and let contain those columns. Its ordinary least squares estimator sets the omitted coefficients to zero and has
For , use . Full column rank of implies full column rank for each . The matrix is an orthogonal projection matrix of rank .
Put and . Projection orthogonality gives the bias-variance decomposition of mean squared error
while the expected residual sum of squares is
These equations distinguish the mean-vector prediction risk in this problem from the additional noise in predicting a new response vector.
First take . Let project onto the full column space of . Since the full model contains , the residual estimate of Gaussian noise variance
is unbiased: its numerator has expectation . Consequently an unbiased Gaussian projection risk estimate is
Indeed its expectation is for every , even when the subset model omits true effects. Independence of the two terms is not needed for this expectation calculation. The full-model variance estimate is important: the subset residual variance generally includes omitted-variable bias.
For a finite candidate collection, choose a model minimizing , breaking ties in favour of a smaller model. The term is common to all models, so this amounts to minimizing
the Mallows Cp form. The fit improvement competes with an increasing dimension penalty. Unbiasedness holds for each fixed model; the estimate at the data-selected model need not remain unbiased, because selection favours downward fluctuations. The method is a heuristic model selection rule, rather than an assertion that it finds the true model with certainty.
The printed includes a genuine qualification. If , there are no full-model residual degrees of freedom. Except in the balanced case , no integrable data-only unbiased estimator can satisfy the requested identity for all means and unknown variances. Here is a proof of the unknown-variance risk estimation in a saturated Gaussian model obstruction. Since is invertible, ranges over all . Suppose had the required expectation at every and variance . For , let and let . Marginally . Applying the assumed identity conditionally gives
whereas applying it to the marginal distribution gives . Their difference is , which must vanish. Integrability under the marginal Gaussian justifies conditioning. If , the obstruction disappears and the residual sum of squares alone has expectation . Thus
The general construction therefore requires , or an independently available unbiased variance estimate.
For the normal linear model, the log-likelihood is
The full column rank of the design matrix makes invertible. Differentiating in gives the normal equations . With the minimized residual sum of squares substituted, differentiation in gives
These are the maximum-likelihood estimators almost surely; almost surely when and . The linear transformation of the multivariate normal distribution yields
Define the hat matrix . The fitted values are , the regression residuals are , and . The orthogonal projection has rank and annihilates . By Cochran's theorem,
Consequently , with bias . The unbiased estimator is
Its square root, the reported residual standard error, estimates ; the assertion of unbiasedness applies to the variance, not generally to its square root.
For the paper-strength analysis, let be the measured percentage and . Both normal linear models use independent errors of common variance . Their mean functions are for lm1, and for lm2, with separately fitted coefficients. The reported residual standard errors are and , respectively.
For lm1, the estimated conditional expectation at a new percentage is
The original data mean must be used for centering the new percentage; its numerical value is not supplied in the excerpt. Because , the two columns of this design matrix are orthogonal, and . Hence the coefficient standard errors in the output give
This estimates uncertainty in the mean. Predicting an individual future batch would additionally require the new-error variance, estimated by .
To compare the nested normal linear models, test against within the quadratic model. The Student t-test statistic is
Its two-sided p-value is . Equivalently is a partial F-test statistic with null distribution . Reject the linear restriction and prefer lm2 at the 5% level. Its residual spread is much smaller; the increase in the coefficient of determination from to supports the same conclusion, although the test is the relevant complexity-adjusted comparison.
For regression diagnostics, inspect regression residuals against fitted means and hardwood percentage for omitted curvature, a scale-location plot for nonconstant variance, and a quantile-quantile plot against the normal distribution for departures from the error assumption. Check unusual observations using regression leverage and Cook's distance, and examine residuals in collection or batch order if dependence is plausible. Independence and a common variance require substantive justification as well as these plots; a small p-value for a polynomial term does not itself check the error model.
Let be speed and the tool type. Each fit is a normal linear model with independent errors. The three mean specifications are
The constraints are treatment coding with type 1 as reference. The parameter counts are , respectively. Model 2 gives parallel lines; model 3 allows interaction terms and distinct slopes.
In the analysis of variance interaction row, adding three independent slope differences costs three degrees of freedom. Its extra sum of squares is the reduction in residual sum of squares, . Its mean square is , and its F-test statistic is
Thus the four missing entries are , , , and , to the precision allowed by the rounded output. This tests against the alternative that at least one slope difference is nonzero. Under and the normal linear model assumptions, the statistic has distribution . Its p-value gives no reason to reject at 5%.
The simpler common-line model is inadequate compared with the parallel-line model. Testing against at least one nonzero type effect, while retaining speed, gives the partial F-test
Its p-value is far below , so reject the common-line restriction. This reduction differs from the sequential type sum of squares in the displayed table, because that table adds type before speed. Recommend model 2: different intercepts and a common decreasing slope.
The selected coefficient estimates give
The intercept estimates the expected lifetime of type 1 at speed zero; if zero speed is outside the data range, it is only an extrapolated intercept. Its standard error is , and its Student t-test ratio is , with two-sided p-value . At any fixed speed, type 2 differs from type 1 by hours, with standard error , and p-value , giving little evidence of a difference. Types 3 and 4 exceed type 1 by and hours, with standard errors and ; their test ratios and and p-values and support positive differences. Each of these individual coefficient tests has null value zero, alternative nonzero, and null distribution .
The speed coefficient means a reduction of hours per extra revolution per minute, or hours per additional 100 rpm, for every type. Its standard error is ; the ratio has null distribution when the common slope is zero, and two-sided p-value .
The residual standard error estimates the common noise standard deviation in hours using 15 residual degrees of freedom, since . The coefficient of determination means that about of the corrected lifetime variation is explained by this fit. The overall F-test statistic tests the simultaneous restriction against at least one nonzero coefficient; its null distribution is and its p-value rejects an intercept-only mean. These conclusions remain conditional on suitable regression diagnostics for independent, homoscedastic, approximately normal errors.
The required sketch plots the four fitted lines. Its speed interval is illustrative because the observed speeds are not supplied; the ordering and vertical gaps are determined by the fitted coefficients.
Figure 1.
Parallel fitted tool-lifetime lines for the four tool types
. Details.
The ordinary least squares objective has gradient . The normal equation therefore gives
The Gram matrix is a positive-definite matrix because has full column rank. More explicitly, for any ,
since . This proves that the displayed solution is the unique global minimum.
The fitted values are , where the hat matrix is
The inverse of the symmetric matrix is symmetric, so . Multiplication gives . Thus is the orthogonal projection onto the column space of the design matrix.
The regression residual vector is with . Consequently , , and the residual sum of squares is
By fitted-residual orthogonality, . Using , the cross-covariance matrix is
Both vectors are jointly normal, so they are also independent by independence of uncorrelated jointly normal variables. The zero covariance calculation itself only needs the common error variance and lack of error correlations.
For the first wind fit, let be wind velocity and electrical output. The simple linear regression is , with independent . It has two fitted mean parameters and residual degrees of freedom. The missing analysis of variance entries are therefore velocity degrees of freedom , velocity mean square , and residual degrees of freedom . The residual mean square is , and , consistent with the printed value after rounding.
Because the model includes a regression intercept, the coefficient of determination is the explained sum of squares divided by the total centered sum of squares:
A large coefficient of determination does not rule out a wrong mean function. In the PDF's first residual-versus-fitted plot, residuals are negative at both ends and positive in the middle. This curved pattern agrees with the visibly flattening output-versus-velocity relationship and motivates a polynomial regression with a quadratic term.
The second wind model is , again with independent common-variance normal errors. Its fitted mean is . The line marked (A) is a two-sided Student t-test of against , conditional on retaining the intercept and linear term. Under the null hypothesis,
Its two-sided p-value is . This is far below , so reject a purely linear mean in favour of the quadratic fit. Equivalently, the extra-term nested-model F-test has and null law .
The third fit is a reciprocal-predictor regression, , with estimated mean . It has two mean parameters rather than the quadratic model's three. Its residual standard error is smaller, versus , and its coefficient of determination is larger, versus . The corresponding residual sums of squares are approximately and . The reciprocal fit is preferable on both these fit measures and parsimony. The two models are not nested, so an ordinary extra-term F-test between them is inappropriate. On the common normal-error likelihood, the difference also favours the third model.
Neither lower residual sum of squares nor a higher coefficient of determination establishes adequate assumptions. The quadratic residual-versus-fitted plot removes the original pronounced curvature; the reciprocal plot also has no comparably obvious mean trend. The low-output residuals appear somewhat more spread out, so check scale-location plots and residuals against velocity for heteroscedasticity. Q-Q plots assess normality; residuals against observation order assess serial dependence; regression leverage and Cook's distance identify influential observations. Further cross-validation, replicate observations at comparable velocities, and prediction errors would help choose between their extrapolation behaviours. Both fitted shapes should be judged principally over the observed positive-velocity range.
Let be recall for age level , processing level , and replicate . Write for Younger and for Older. With Older and A as the reference levels in a regression factor, the additive two-factor normal linear model under treatment coding is
where the errors are independent , with common positive variance. The design matrix is fixed; subjects supply independent observations; the specified conditional mean is correct; and the absence of an interaction term means the age difference is constant across processing levels. These are model assumptions, not consequences of random assignment. The treatment allocation supports independence between processing assignment and background characteristics, but age itself was not randomized.
An older subject in A has and , so the estimated mean is words. Its standard error is the intercept's . There are residual degrees of freedom, giving the 95% confidence interval for this mean:
This is a mean-response confidence interval, not a prediction interval for a new individual's recall; the latter also includes the new observation's error variance.
The second two-factor normal linear model adds age-by-processing interaction terms:
Its ten free mean parameters are equivalent to one mean for each of the ten cells. To compare the fits, the null hypothesis is . The full residual sum of squares is on degrees of freedom. Adding interaction reduces the additive model's residual sum of squares by , so the reduced value is . The nested-model F-test statistic is
Its p-value is . Reject additivity and retain the interaction model. The significant interaction means that a single age effect averaged over all processing methods does not adequately summarize the data.
The regression intercept is the older-A mean. The age coefficient is the younger-minus-older contrast specifically in A. The processing coefficients compare B, C, D and E with A specifically among older subjects. The interaction coefficients are differences between the age contrast in each of those methods and its value in A. Adding the relevant coefficients gives
Thus A, C and especially D favour younger subjects in their estimated means, whereas B and E have little estimated age difference. In the younger group, D has the largest estimated recall, followed by C and A; B and E are much lower. Among older subjects C is largest, D and A are intermediate, and B and E are lowest. Comparing these orders is a description of estimates, not a claim that every pairwise difference is significant.
Each coefficient's printed standard error estimates its uncertainty; its statistic divides the estimate by that error, and its two-sided p-value uses under the relevant zero-contrast null hypothesis. For example, the older B-versus-A and E-versus-A contrasts have and . The C-versus-A and D-versus-A older contrasts have and . The B interaction has , while the D interaction is borderline at and the other two individual interaction tests are not significant at . These individual tests do not override the joint interaction F-test, and multiple comparisons require care. A simple younger-versus-older contrast outside A combines coefficients and must use their estimated covariance, rather than adding their marginal errors.
The residual standard error estimates . The coefficient of determination is the fraction of centered sample recall variation explained by the ten-cell model. The overall with null law tests all nine non-intercept coefficients jointly against zero, giving extremely strong evidence that the cell means are not all equal. The sequential analysis of variance separates age, process, and their interaction; the balanced design makes the main-effect sums orthogonal, whereas the coefficient table uses the specified reference-level contrasts.
To assess pooling into three processing types, retain age-by-type interaction because the previous test rejects additivity. The reduced model has six cell means. Its null hypothesis says A and C have equal means within each age, and B and E have equal means within each age: four restrictions. Compare it with the ten-cell model using
In fact the supplied fitted cell means and equal cell sizes determine the increase without raw observations. Pooling two ten-person cells with means adds to the residual sum of squares. Here the four differences are , so
Therefore and . At , the data do not reject the three-type reduction, though the result is borderline and is not proof that the pooled means are identical. This factor-level pooling test keeps the previously supported interaction structure; reducing the categories and removing all interaction simultaneously would test a different set of restrictions.