For the normal linear model, the log-likelihood is
The full column rank of the design matrix makes invertible. Differentiating in gives the normal equations . With the minimized residual sum of squares substituted, differentiation in gives
These are the maximum-likelihood estimators almost surely; almost surely when and . The linear transformation of the multivariate normal distribution yields
Define the hat matrix . The fitted values are , the regression residuals are , and . The orthogonal projection has rank and annihilates . By Cochran's theorem,
Consequently , with bias . The unbiased estimator is
Its square root, the reported residual standard error, estimates ; the assertion of unbiasedness applies to the variance, not generally to its square root.
For the paper-strength analysis, let be the measured percentage and . Both normal linear models use independent errors of common variance . Their mean functions are for lm1, and for lm2, with separately fitted coefficients. The reported residual standard errors are and , respectively.
For lm1, the estimated conditional expectation at a new percentage is
The original data mean must be used for centering the new percentage; its numerical value is not supplied in the excerpt. Because , the two columns of this design matrix are orthogonal, and . Hence the coefficient standard errors in the output give
This estimates uncertainty in the mean. Predicting an individual future batch would additionally require the new-error variance, estimated by .
To compare the nested normal linear models, test against within the quadratic model. The Student t-test statistic is
Its two-sided p-value is . Equivalently is a partial F-test statistic with null distribution . Reject the linear restriction and prefer lm2 at the 5% level. Its residual spread is much smaller; the increase in the coefficient of determination from to supports the same conclusion, although the test is the relevant complexity-adjusted comparison.
For regression diagnostics, inspect regression residuals against fitted means and hardwood percentage for omitted curvature, a scale-location plot for nonconstant variance, and a quantile-quantile plot against the normal distribution for departures from the error assumption. Check unusual observations using regression leverage and Cook's distance, and examine residuals in collection or batch order if dependence is plausible. Independence and a common variance require substantive justification as well as these plots; a small p-value for a polynomial term does not itself check the error model.
Let be speed and the tool type. Each fit is a normal linear model with independent errors. The three mean specifications are
The constraints are treatment coding with type 1 as reference. The parameter counts are , respectively. Model 2 gives parallel lines; model 3 allows interaction terms and distinct slopes.
In the analysis of variance interaction row, adding three independent slope differences costs three degrees of freedom. Its extra sum of squares is the reduction in residual sum of squares, . Its mean square is , and its F-test statistic is
Thus the four missing entries are , , , and , to the precision allowed by the rounded output. This tests against the alternative that at least one slope difference is nonzero. Under and the normal linear model assumptions, the statistic has distribution . Its p-value gives no reason to reject at 5%.
The simpler common-line model is inadequate compared with the parallel-line model. Testing against at least one nonzero type effect, while retaining speed, gives the partial F-test
Its p-value is far below , so reject the common-line restriction. This reduction differs from the sequential type sum of squares in the displayed table, because that table adds type before speed. Recommend model 2: different intercepts and a common decreasing slope.
The selected coefficient estimates give
The intercept estimates the expected lifetime of type 1 at speed zero; if zero speed is outside the data range, it is only an extrapolated intercept. Its standard error is , and its Student t-test ratio is , with two-sided p-value . At any fixed speed, type 2 differs from type 1 by hours, with standard error , and p-value , giving little evidence of a difference. Types 3 and 4 exceed type 1 by and hours, with standard errors and ; their test ratios and and p-values and support positive differences. Each of these individual coefficient tests has null value zero, alternative nonzero, and null distribution .
The speed coefficient means a reduction of hours per extra revolution per minute, or hours per additional 100 rpm, for every type. Its standard error is ; the ratio has null distribution when the common slope is zero, and two-sided p-value .
The residual standard error estimates the common noise standard deviation in hours using 15 residual degrees of freedom, since . The coefficient of determination means that about of the corrected lifetime variation is explained by this fit. The overall F-test statistic tests the simultaneous restriction against at least one nonzero coefficient; its null distribution is and its p-value rejects an intercept-only mean. These conclusions remain conditional on suitable regression diagnostics for independent, homoscedastic, approximately normal errors.
The required sketch plots the four fitted lines. Its speed interval is illustrative because the observed speeds are not supplied; the ordering and vertical gaps are determined by the fitted coefficients.
Figure 1.
Parallel fitted tool-lifetime lines for the four tool types
.
The ordinary least squares objective has gradient . The normal equation therefore gives
The Gram matrix is a positive-definite matrix because has full column rank. More explicitly, for any ,
since . This proves that the displayed solution is the unique global minimum.
The fitted values are , where the hat matrix is
The inverse of the symmetric matrix is symmetric, so . Multiplication gives . Thus is the orthogonal projection onto the column space of the design matrix.
The regression residual vector is with . Consequently , , and the residual sum of squares is
By fitted-residual orthogonality, . Using , the cross-covariance matrix is
Both vectors are jointly normal, so they are also independent by independence of uncorrelated jointly normal variables. The zero covariance calculation itself only needs the common error variance and lack of error correlations.
For the first wind fit, let be wind velocity and electrical output. The simple linear regression is , with independent . It has two fitted mean parameters and residual degrees of freedom. The missing analysis of variance entries are therefore velocity degrees of freedom , velocity mean square , and residual degrees of freedom . The residual mean square is , and , consistent with the printed value after rounding.
Because the model includes a regression intercept, the coefficient of determination is the explained sum of squares divided by the total centered sum of squares:
A large coefficient of determination does not rule out a wrong mean function. In the PDF's first residual-versus-fitted plot, residuals are negative at both ends and positive in the middle. This curved pattern agrees with the visibly flattening output-versus-velocity relationship and motivates a polynomial regression with a quadratic term.
The second wind model is , again with independent common-variance normal errors. Its fitted mean is . The line marked (A) is a two-sided Student t-test of against , conditional on retaining the intercept and linear term. Under the null hypothesis,
Its two-sided p-value is . This is far below , so reject a purely linear mean in favour of the quadratic fit. Equivalently, the extra-term nested-model F-test has and null law .
The third fit is a reciprocal-predictor regression, , with estimated mean . It has two mean parameters rather than the quadratic model's three. Its residual standard error is smaller, versus , and its coefficient of determination is larger, versus . The corresponding residual sums of squares are approximately and . The reciprocal fit is preferable on both these fit measures and parsimony. The two models are not nested, so an ordinary extra-term F-test between them is inappropriate. On the common normal-error likelihood, the difference also favours the third model.
Neither lower residual sum of squares nor a higher coefficient of determination establishes adequate assumptions. The quadratic residual-versus-fitted plot removes the original pronounced curvature; the reciprocal plot also has no comparably obvious mean trend. The low-output residuals appear somewhat more spread out, so check scale-location plots and residuals against velocity for heteroscedasticity. Q-Q plots assess normality; residuals against observation order assess serial dependence; regression leverage and Cook's distance identify influential observations. Further cross-validation, replicate observations at comparable velocities, and prediction errors would help choose between their extrapolation behaviours. Both fitted shapes should be judged principally over the observed positive-velocity range.
Let be recall for age level , processing level , and replicate . Write for Younger and for Older. With Older and A as the reference levels in a regression factor, the additive two-factor normal linear model under treatment coding is
where the errors are independent , with common positive variance. The design matrix is fixed; subjects supply independent observations; the specified conditional mean is correct; and the absence of an interaction term means the age difference is constant across processing levels. These are model assumptions, not consequences of random assignment. The treatment allocation supports independence between processing assignment and background characteristics, but age itself was not randomized.
An older subject in A has and , so the estimated mean is words. Its standard error is the intercept's . There are residual degrees of freedom, giving the 95% confidence interval for this mean:
This is a mean-response confidence interval, not a prediction interval for a new individual's recall; the latter also includes the new observation's error variance.
The second two-factor normal linear model adds age-by-processing interaction terms:
Its ten free mean parameters are equivalent to one mean for each of the ten cells. To compare the fits, the null hypothesis is . The full residual sum of squares is on degrees of freedom. Adding interaction reduces the additive model's residual sum of squares by , so the reduced value is . The nested-model F-test statistic is
Its p-value is . Reject additivity and retain the interaction model. The significant interaction means that a single age effect averaged over all processing methods does not adequately summarize the data.
The regression intercept is the older-A mean. The age coefficient is the younger-minus-older contrast specifically in A. The processing coefficients compare B, C, D and E with A specifically among older subjects. The interaction coefficients are differences between the age contrast in each of those methods and its value in A. Adding the relevant coefficients gives
Thus A, C and especially D favour younger subjects in their estimated means, whereas B and E have little estimated age difference. In the younger group, D has the largest estimated recall, followed by C and A; B and E are much lower. Among older subjects C is largest, D and A are intermediate, and B and E are lowest. Comparing these orders is a description of estimates, not a claim that every pairwise difference is significant.
Each coefficient's printed standard error estimates its uncertainty; its statistic divides the estimate by that error, and its two-sided p-value uses under the relevant zero-contrast null hypothesis. For example, the older B-versus-A and E-versus-A contrasts have and . The C-versus-A and D-versus-A older contrasts have and . The B interaction has , while the D interaction is borderline at and the other two individual interaction tests are not significant at . These individual tests do not override the joint interaction F-test, and multiple comparisons require care. A simple younger-versus-older contrast outside A combines coefficients and must use their estimated covariance, rather than adding their marginal errors.
The residual standard error estimates . The coefficient of determination is the fraction of centered sample recall variation explained by the ten-cell model. The overall with null law tests all nine non-intercept coefficients jointly against zero, giving extremely strong evidence that the cell means are not all equal. The sequential analysis of variance separates age, process, and their interaction; the balanced design makes the main-effect sums orthogonal, whereas the coefficient table uses the specified reference-level contrasts.
To assess pooling into three processing types, retain age-by-type interaction because the previous test rejects additivity. The reduced model has six cell means. Its null hypothesis says A and C have equal means within each age, and B and E have equal means within each age: four restrictions. Compare it with the ten-cell model using
In fact the supplied fitted cell means and equal cell sizes determine the increase without raw observations. Pooling two ten-person cells with means adds to the residual sum of squares. Here the four differences are , so
Therefore and . At , the data do not reject the three-type reduction, though the result is borderline and is not proof that the pooled means are identical. This factor-level pooling test keeps the previously supported interaction structure; reducing the categories and removing all interaction simultaneously would test a different set of restrictions.
The statistician first models the age- and gender-dependent mean by linear regression. Its fitted value is . At fixed gender, a year of age is associated with a decrease of BMI units; the group coded gender has a fitted mean units above the group coded at the same age. This question does not state which gender receives which code. The intercept refers to age zero in the reference group, outside a typical adult-patient range, so its substantive interpretation is limited. Both slope Student t-tests have small p-values, and the overall F-test supports an age/gender-dependent mean. The coefficient of determination is only , leaving considerable individual variation to investigate.
The residual diagnostics check whether mean adjustment is plausible and whether a single normal error law is adequate. The residual-versus-fitted plot has no strong curved mean trend, although its smooth curve is not perfectly flat. The scale-location plot suggests some decline in spread as fitted BMI increases, so common residual variance is a working approximation rather than an established fact. The normal Q-Q plot has systematic departures, notably shorter tails than the normal reference, suggesting the residual law is not exactly normal. The regression leverage plot does not display an obviously extreme leverage point, but the marked observations and Cook's distance should still be checked for influence. Independence between patients cannot be diagnosed from these four plots alone.
Next the statistician extracts the regression residuals and fits an intercept-only normal model as a baseline residual distribution. With an intercept in the original ordinary least squares model, the residuals sum to zero by the normal equation; the near-zero fitted residual mean and its are therefore automatic, not new evidence of good fit. The standard error in this refit uses degrees of freedom, whereas the original uses after estimating three regression coefficients. The reported log-likelihood for the single-normal residual model is .
The histogram suggests a shape worth exploring beyond one normal density, with broad shoulders and some asymmetry, without showing unambiguous separated clusters. The statistician fits two- and three-component finite Gaussian mixtures with a common variance by the expectation-maximization algorithm, starting from separated means and positive weights. Successive likelihoods increase and the displayed final iterations stabilize, as expected from EM likelihood monotonicity. This is evidence of numerical convergence from those starts, not proof of a global maximum.
The two-component fit assigns weights , residual means and common variance . The three-component fit assigns weights , residual means and common variance . The code correctly passes the square roots of those variances to dnorm and overlays the resulting weighted densities. Both curves broadly follow the histogram and are very similar. In particular, a two-component mixture need not have two distinct visible modes.
These fits explore residual mixture clustering: groups differ in BMI relative to the same age/gender-adjusted mean, rather than simply in raw BMI. Membership can be summarized by the fitted mixture responsibilities, retaining uncertainty instead of asserting a certain label for each patient. The common-variance assumption is economical, but should be checked; the common regression slope assumption and constancy of mixture proportions across age and gender are also substantive. A residual mixture alone does not establish genuine biological subpopulations; a flexible continuous distribution, missing predictors, nonlinear mean effects or heteroscedasticity could explain a similar marginal shape.
There is also a two-stage residual mixture fitting limitation. Fitted regression residuals are not exactly independent identically distributed observations: even under a normal regression, their covariance matrix is , and estimating the mean adjustment introduces uncertainty. With subjects and three initial regression coefficients, treating them as independent is a useful approximation for exploring shape, but ordinary mixture likelihoods omit the first-stage uncertainty. A joint mixture regression with shared slopes, or a suitable bootstrap of the whole analysis, would give a firmer basis for parameter uncertainty and cluster selection. Examine component membership versus age/gender and repeat the algorithm from multiple starts before making substantive clustering claims.
The command fits the simple linear regression
using ordinary least squares. The fitted conditional mean is a straight line in time. The regression residuals are mainly positive at the beginning and end and negative in the middle, so the mean function needs curvature.
A direct improvement is polynomial regression of degree two:
fit2 <- lm(Counts ~ Time + I(Time^2), data = geiger)
anova(fit1, fit2)
The nested-model F-test tests against a nonzero quadratic coefficient. If measurements were used, its statistic is
A small reported -value supports the quadratic term. The new regression residuals should then be checked for remaining systematic structure, changing variance and temporal dependence, because the F-test assumes independent errors of common variance. Radioactive decay also motivates an exponential mean, but the displayed curvature already justifies this simple nested comparison.
Write . Minimizing the residual sum of squares first over the coefficients of reduces the problem to minimizing . Full column rank ensures . Thus the partial regression formula and its variance are
This uses and the isotropic error covariance matrix .
Since the span of is contained in the column space of , orthogonal projection onto the latter removes at least as much squared norm. Consequently
and inversion proves the requested lower bound. Full rank makes the denominator positive.
Adding to column leaves the column space unchanged. Every old fitted vector is reproduced by keeping all slopes unchanged and replacing the regression intercept by . Uniqueness of the ordinary least squares estimators then proves that slope is unchanged. In particular all columns can be centered by such changes. Applying the previous projection argument to the centered design proves the bound with and ; the normalized inner product is now the usual sample correlation.
Each reported coefficient test is the two-sided Student's t-test of against , conditional on the other predictors. Under the null, has a Student's t-distribution with degrees of freedom. At five percent the intercept is significant, but neither individual test slope is. The final F-test compares the full model against an intercept-only model: . It has degrees of freedom and rejects at five percent, since its -value is . Thus the predictors jointly explain variation, without strong evidence separating their individual conditional effects.
The two test predictors have sample correlation , although each has only moderate correlation with the final exam response. The variance inflation factor is . Their nearly shared direction produces substantial multicollinearity, inflating the individual slope standard errors. The joint F-test can detect this common predictive direction even though either coefficient, given the other, has a large -value.
In the larger normal linear model, add laboratory intercept offsets to the mean in part (a). The nested-model F-test compares
The smaller and larger models fit four and six mean coefficients respectively. Under , independent Gaussian errors with common variance give independent quantities and . Thus the test statistic is
The p-value is the upper-tail probability . At the conventional 5% significance level, do not reject the common-intercept restriction. Prefer the simpler model tumour1 on this comparison. This does not prove the intercepts coincide, nor address the curvature identified in part (b); the exact F-test calibration assumes the underlying error and mean assumptions. The one-decimal RSS values give when subtracted, whereas the table's difference uses unrounded values.
Check that the chosen mean structure adequately describes the transformed response. Examine the residual-versus-fitted plot and residuals versus each predictor: systematic curvature suggests omitted nonlinear effects or an interaction term. Plot residual spread against fitted values, for example using a scale-location plot, to assess homoscedasticity. Compare standardized residuals with a normal distribution using a quantile-quantile plot, especially for finite-sample Student's t-distribution and F-test inference.
Also check independence using the sampling design and residuals versus time, collection site, or other groups; spatially related photovoltaic systems may have correlated errors that are invisible in a residual-versus-fitted plot. Investigate large standardized regression residuals, high regression leverage, and influential observations using Cook's distance. Check that the design matrix has full rank and that severe multicollinearity is not making estimates unstable. Reassess whether the response transformation and retained predictors improve these diagnostics and prediction; a coefficient p-value alone cannot establish model adequacy. If observations are independent only conditional on site effects, use an appropriate dependence model rather than treating correlated systems as independent replicates.
Here the full F-test has and . There is no evidence against the joint null hypothesis that both slopes vanish, and hence no evidence in this model that either covariate explains the response.
The full F-test has and tests the null hypothesis that both slope coefficients vanish. It decisively rejects that hypothesis, while says the fitted model explains about of the sample variation. Together with the insignificant individual t-tests, this is characteristic of multicollinearity: the two covariates explain the response jointly, but the output does not identify either one as having a separately estimable effect.
Polynomial regression 2026-10-06
A linear regression in the coefficients with a polynomial mean function . Successive degrees define nested models that can be compared by an F-test.
Residual deviance 2026-10-06
The unscaled residual deviance of an exponential dispersion family compares its fitted mean with the saturated statistical model at a common dispersion parameter. Its normalization removes the factor in the log-likelihood. In Poisson regression and binomial regression, , so it is twice the saturated-minus-fitted log-likelihood. Under suitable regularity, has approximately a chi-squared distribution with the residual degrees of freedom; this is not an automatic accurate approximation for sparse responses. Nested deviance reductions with estimated dispersion lead to approximate F-tests after scaling by the dispersion estimate.