Factor-level pooling test 2026-10-06
To pool factor levels in a normal linear model, impose equality of their fitted means while retaining any needed interaction terms. Compare the resulting nested model with the full model using a nested-model F-test. For two cells each of size , pooling their fitted means adds to the residual sum of squares. If pooling two pairs separately within two other-factor levels, there are four restrictions.
Past exam of the mathematics course of the University of Cambridge 2006 iii Paper 41 2 Solution Created 2026-10-03 Updated 2026-10-14
A factor is a categorical predictor whose levels distinguish experimental conditions. Two factors have a statistical interaction when the effect of changing one depends on the other's level. For cell means , absence of an interaction means ; equivalently, is independent of .
Index thickness by , temperature by , wash by and replication by . The full factorial design has the normal linear modelThe errors are independent with a common positive variance. Sum-to-zero constraints in each factor index, or the displayed treatment coding, identify the mean coefficients. This full model allows an arbitrary mean in each of the sixteen cells.
The missing statistical degrees of freedom, in the output order, are . The first seven entries total fifteen; with the regression intercept there are sixteen mean coefficients among thirty-two observations, leaving sixteen residual statistical degrees of freedom. Balanced factorial orthogonality makes the term sums of squares in ANOVA independent of their ordering.
All terms involving wash have nonsignificant individual F-tests, and it is useful to test their removal jointly. The reduced model omits twelve mean coefficients. Its residual sum of squares is , compared with for the full model, so the nested-model F-test givesThere is no detected wash effect over the four tested durations. Removing its main effect and all its interaction terms respects the principle of marginality. Retaining the very large thickness-by-temperature interaction is essential: the thickness effect changes substantially with temperature.
With numeric indicators , treatment coding expresses the reduced fitted mean asThus the thin/low, thick/low, thin/high and thick/high fitted means are , respectively, at every wash level. Each is the sample mean of eight observations. The equal-cell replication variance formula therefore givesAlternatively, use the variance of a fitted regression mean with : . Individual coefficient standard errors cannot simply be added; their covariances enter the fitted-mean calculation.
Increasing temperature raises fitted lustre by for thin film and for thick film. Increasing thickness raises it by at low temperature and at high temperature. Their difference, , is the estimated interaction contrast; its standard error is , giving on twenty-eight statistical degrees of freedom. The interaction plot shows this strong departure from parallel mean lines, while the wash panel shows the much smaller variation across wash durations. Neither the nonsignificant wash test nor the plot proves that its true effect is exactly zero.
Past exam of the mathematics course of the University of Cambridge 2007 iii Paper 44 1 Solution Created 2026-10-03 Updated 2026-10-14
A useful full statement is the rank-sum form of Cochran's theorem. Let and let be real symmetric matrices satisfying . Put and . Then , and the following conditions are equivalent: the ranks sum to ; the are orthogonal projection matrices with mutually orthogonal ranges; and the are independent, with . A rank-zero form is identically zero. In particular, the usual projection version of Cochran's theorem partitions the squared length of a standard normal vector into independent chi-squared distributions.
Here is a proof which also explains the rank condition. Let . Since every vector satisfies , the spaces span . Therefore their dimensions sum to at least . If they sum to , their sum is a direct sum, so every vector has a unique decomposition into vectors belonging to the . For , the identity is one such decomposition, while the decomposition with just in position and zero elsewhere is another. Uniqueness gives . ThusA symmetric matrix with this idempotence is an orthogonal projection matrix. Its different ranges are orthogonal: for , , their inner product is . Choose an orthonormal basis for each range and concatenate them. The resulting change-of-basis matrix is an orthogonal matrix. In these coordinates, each is the sum of squares of the coordinates belonging to its range.
The transformed multivariate normal distribution still has density proportional to , which factors into the standard normal densities of its coordinates. The coordinates are independent, so the disjoint groups of squared coordinates give independent chi-squared distributions of the asserted degrees of freedom. This proves the forward implication. Conversely, if the have the asserted chi-squared laws, then , while has expectation . Taking expectations yields . This completes the equivalence and the proof of Cochran's theorem.
For the application, specify a normal linear modelwith fixed full-column-rank design matrix of size , , unrestricted , and unknown . Test against , where has independent rows, , and is specified. The null contains an affine space of means of dimension , whereas the full mean space has dimension .
Let be the ordinary least squares and maximum-likelihood estimate in the full model, and write . The constrained estimate follows by minimizing the residual squared length under :Indeed, and the correction is the required constrained projection in the inner product. The least-squares normal equations give the Pythagorean decompositionConsequently the explicit residual quantities in the likelihood-ratio test statistic areHere is the full-model residual sum of squares, and is the increase caused by imposing the null restrictions.
To see the geometry behind Cochran's theorem, choose any with , and let project onto and onto . For ,The projectors and are orthogonal and have ranks and . Under , the mean of belongs to , so both of these projectors annihilate it. Thus Cochran's theorem givesThe nuisance variance cancels in the nested-model F-test statisticSince the likelihood-ratio test statistic isit is a strictly increasing function of . Therefore an exact level- likelihood-ratio test rejects when , where the right-hand side is the upper reference quantile of the F-distribution. This calibration is exact under the normal linear model, rather than an asymptotic chi-squared replacement.
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 38 1 Solution Created 2026-10-03 Updated 2026-10-14
Let be the flight distance in millimetres for paper level , launch-angle level , design level and replicate , where and . The full factorial design is fitted by the normal linear modelThe assumptions are independent errors with a common variance and normal distribution, and the displayed factor-combination mean structure. The corner-point constraints areand whenever at least one index equals . Thus the eight free mean coefficients are . These treatment coding coefficients are comparisons at the other factors' reference levels, rather than necessarily averaged main effects.
To test the three-factor interaction term, compare the full model with the model retaining all main effects and all two-factor interactions. The null hypothesis is , against a nonzero value. Removing it increases the residual sum of squares by and removes one parameter. With eight residual degrees of freedom, the nested-model F-test givesThere is no evidence that this three-factor interaction is needed. This does not establish that all two-factor interactions vanish: the paper-design interaction is substantial.
The
stepAIC command performs stepwise selection by the Akaike information criterion. Its upper scope is the full factorial model and its lower scope is the intercept-only model. Starting from the full model, it compares admissible additions and deletions, preserving the hierarchy of interaction terms, and chooses changes that reduce the Akaike information criterion. With the supplied scope its default search permits both directions. For these Gaussian fits the displayed criterion, up to constants common to all models, is , where counts mean coefficients. The first deletion changes it from to : the small increase in residual sum of squares is outweighed by removing a parameter. This is a greedy model selection procedure, not an exhaustive search or a sequence of fixed-level hypothesis tests. The command's arguments are described in the MASS documentation.For the selected model, let and . Thenindependently between flights. It has no angle term. The maximum-likelihood estimates of its mean coefficients are also obtained by ordinary least squares:For example, the four fitted means are just the four paper-design cell means, each averaging four flights over the angle levels. With , the variance-covariance matrix of the coefficient estimates is . Its diagonal gives the displayed standard errors; their ratios are , with mm.
The final line of
summary(model2.lm) is the joint nested-model F-test of , comparing this four-parameter model with a common mean. Its statistic iswith F-distribution under the fixed-model null and P-value . Thus the selected paper-design terms jointly improve the fit substantially. The coefficient of determination means they account for about of the observed centred variation. These nominal inferential summaries do not adjust for the preceding model selection on the same data.Using the unrounded fitted means givesin millimetres. Hence choose the high-performance design with the lighter paper; the fitted model leaves both launch angles tied, with expected distance about mm. A preference for one angle is not supported by this selected model. The strong paper-design interaction reverses the design comparison: the simple design is better on heavy paper, whereas the high-performance design is better on light paper. Consequently a single design effect averaged over paper types would conceal the main result. There is little evidence here for an angle effect, and considerable flight-to-flight variability remains.
Before fitting, inspect raw-distance dot plots for all eight factor combinations, and interaction plots of cell means against design, with separate lines for paper and panels for angle. With only two replicates per cell, showing individual observations is more informative than elaborate box plots. These reveal reversals, spread and possible unusual flights. Afterwards inspect a residual-versus-fitted plot, a Q-Q plot of residuals, a scale-location plot, residuals against each factor and, if recorded, flight order, and Cook's distance for influential observations. Residual patterns by angle would question its deletion, while unequal spread or departures from the normal distribution would question the error assumptions rather than merely demand another mean term.
Past exam of the mathematics course of the University of Cambridge 2012 iii Paper 37 2 Solution Created 2026-10-03 Updated 2026-10-07
A regression factor is a categorical predictor represented by level indicators and contrasts. Treating year as a regression factor permits arbitrary differences between the three yearly means, reflecting conditions such as weather, rather than imposing a linear time trend.
Let be the yield for seed variety , fertiliser level , and year . A treatment-coded two-factor normal linear model with additive year effects iswith independent errors. The unrestricted coefficients are : six parameters for twelve observations. Equivalently use baseline-zero variety, fertiliser, and interaction arrays. There are no year-treatment interactions in this fit.
Dividing each sum of squares by its statistical degrees of freedom gives the completed analysis of variance:In particular, the missing interaction statistic is . Its null hypothesis is , against : the change in mean yield from high fertiliser is the same for both varieties. Under the null hypothesis and the independent homoscedastic normal linear model, this nested-model F-test has distribution . The printed -value 0.001137 strongly rejects no interaction.
Do not drop fertiliser just because its main-effect ANOVA row has . In this balanced factorial design that row measures a fertiliser effect averaged over varieties; large opposite effects can cancel. The strongly supported interaction requires retaining its associated main effects by regression model hierarchy. Seed and year also have evidence of effects in their ANOVA rows, so this table alone gives no compelling simplification of the fitted terms.
The interaction contrast is , with standard error 0.7946. The fitted high-minus-low fertiliser effect is for variety 1 but for variety 2. The fitted variety-2-minus-variety-1 contrast is under low fertiliser and under high fertiliser. Thus variety 2 performs best under low fertiliser, while high fertiliser benefits variety 1 and reduces variety 2's fitted yield. The marginal fertiliser effect is only .
The four fitted means in the baseline year are for respectively. Add in 2005 and in 2006 to every fitted mean. The 2005 coefficient has under a test of a zero contrast with 2004; the analogous 2006 contrast has . The latter is lack of evidence for a difference, not proof that the yearly means coincide. Standard errors for the combined contrasts require the coefficient covariance matrix. The experiment is small: residual standard error is 0.6882 on only six statistical degrees of freedom. If the same fields were reused, possible within-field dependence and field allocation would need checking before treating the fitted error independence as established.
Past exam of the mathematics course of the University of Cambridge 2013 iii Paper 30 2 d Solution Created 2026-10-03 Updated 2026-10-07
Test for both nonreference days, against at least one nonzero day fixed effect, conditional on chocolate. There are two added regression coefficients and residual statistical degrees of freedom. The nested-model F-test statistic isUnder and the normal linear model assumptions, . Its 5% upper critical value is , so do not reject the absence of day effects; the p-value is approximately . Thus the missing row has two statistical degrees of freedom, mean square , and the F-test value above. These observations do not provide evidence that including day improves the chocolate-adjusted mean model. They do not prove that every possible day effect or chocolate–day interaction term is absent.
Past exam of the mathematics course of the University of Cambridge 2015 iii Paper 33 1 Solution Created 2026-10-03 Updated 2026-10-06
The ordinary least squares objective has gradient . The normal equation therefore givesThe Gram matrix is a positive-definite matrix because has full column rank. More explicitly, for any ,since . This proves that the displayed solution is the unique global minimum.
The fitted values are , where the hat matrix isThe inverse of the symmetric matrix is symmetric, so . Multiplication gives . Thus is the orthogonal projection onto the column space of the design matrix.
The regression residual vector is with . Consequently , , and the residual sum of squares isBy fitted-residual orthogonality, . Using , the cross-covariance matrix isBoth vectors are jointly normal, so they are also independent by independence of uncorrelated jointly normal variables. The zero covariance calculation itself only needs the common error variance and lack of error correlations.
For the first wind fit, let be wind velocity and electrical output. The simple linear regression is , with independent . It has two fitted mean parameters and residual degrees of freedom. The missing analysis of variance entries are therefore velocity degrees of freedom , velocity mean square , and residual degrees of freedom . The residual mean square is , and , consistent with the printed value after rounding.
Because the model includes a regression intercept, the coefficient of determination is the explained sum of squares divided by the total centered sum of squares:A large coefficient of determination does not rule out a wrong mean function. In the PDF's first residual-versus-fitted plot, residuals are negative at both ends and positive in the middle. This curved pattern agrees with the visibly flattening output-versus-velocity relationship and motivates a polynomial regression with a quadratic term.
The second wind model is , again with independent common-variance normal errors. Its fitted mean is . The line marked (A) is a two-sided Student t-test of against , conditional on retaining the intercept and linear term. Under the null hypothesis,Its two-sided p-value is . This is far below , so reject a purely linear mean in favour of the quadratic fit. Equivalently, the extra-term nested-model F-test has and null law .
The third fit is a reciprocal-predictor regression, , with estimated mean . It has two mean parameters rather than the quadratic model's three. Its residual standard error is smaller, versus , and its coefficient of determination is larger, versus . The corresponding residual sums of squares are approximately and . The reciprocal fit is preferable on both these fit measures and parsimony. The two models are not nested, so an ordinary extra-term F-test between them is inappropriate. On the common normal-error likelihood, the difference also favours the third model.
Neither lower residual sum of squares nor a higher coefficient of determination establishes adequate assumptions. The quadratic residual-versus-fitted plot removes the original pronounced curvature; the reciprocal plot also has no comparably obvious mean trend. The low-output residuals appear somewhat more spread out, so check scale-location plots and residuals against velocity for heteroscedasticity. Q-Q plots assess normality; residuals against observation order assess serial dependence; regression leverage and Cook's distance identify influential observations. Further cross-validation, replicate observations at comparable velocities, and prediction errors would help choose between their extrapolation behaviours. Both fitted shapes should be judged principally over the observed positive-velocity range.
Past exam of the mathematics course of the University of Cambridge 2015 iii Paper 33 2 Solution Created 2026-10-03 Updated 2026-10-06
Let be recall for age level , processing level , and replicate . Write for Younger and for Older. With Older and A as the reference levels in a regression factor, the additive two-factor normal linear model under treatment coding iswhere the errors are independent , with common positive variance. The design matrix is fixed; subjects supply independent observations; the specified conditional mean is correct; and the absence of an interaction term means the age difference is constant across processing levels. These are model assumptions, not consequences of random assignment. The treatment allocation supports independence between processing assignment and background characteristics, but age itself was not randomized.
An older subject in A has and , so the estimated mean is words. Its standard error is the intercept's . There are residual degrees of freedom, giving the 95% confidence interval for this mean:This is a mean-response confidence interval, not a prediction interval for a new individual's recall; the latter also includes the new observation's error variance.
The second two-factor normal linear model adds age-by-processing interaction terms:Its ten free mean parameters are equivalent to one mean for each of the ten cells. To compare the fits, the null hypothesis is . The full residual sum of squares is on degrees of freedom. Adding interaction reduces the additive model's residual sum of squares by , so the reduced value is . The nested-model F-test statistic isIts p-value is . Reject additivity and retain the interaction model. The significant interaction means that a single age effect averaged over all processing methods does not adequately summarize the data.
The regression intercept is the older-A mean. The age coefficient is the younger-minus-older contrast specifically in A. The processing coefficients compare B, C, D and E with A specifically among older subjects. The interaction coefficients are differences between the age contrast in each of those methods and its value in A. Adding the relevant coefficients givesThus A, C and especially D favour younger subjects in their estimated means, whereas B and E have little estimated age difference. In the younger group, D has the largest estimated recall, followed by C and A; B and E are much lower. Among older subjects C is largest, D and A are intermediate, and B and E are lowest. Comparing these orders is a description of estimates, not a claim that every pairwise difference is significant.
Each coefficient's printed standard error estimates its uncertainty; its statistic divides the estimate by that error, and its two-sided p-value uses under the relevant zero-contrast null hypothesis. For example, the older B-versus-A and E-versus-A contrasts have and . The C-versus-A and D-versus-A older contrasts have and . The B interaction has , while the D interaction is borderline at and the other two individual interaction tests are not significant at . These individual tests do not override the joint interaction F-test, and multiple comparisons require care. A simple younger-versus-older contrast outside A combines coefficients and must use their estimated covariance, rather than adding their marginal errors.
The residual standard error estimates . The coefficient of determination is the fraction of centered sample recall variation explained by the ten-cell model. The overall with null law tests all nine non-intercept coefficients jointly against zero, giving extremely strong evidence that the cell means are not all equal. The sequential analysis of variance separates age, process, and their interaction; the balanced design makes the main-effect sums orthogonal, whereas the coefficient table uses the specified reference-level contrasts.
To assess pooling into three processing types, retain age-by-type interaction because the previous test rejects additivity. The reduced model has six cell means. Its null hypothesis says A and C have equal means within each age, and B and E have equal means within each age: four restrictions. Compare it with the ten-cell model usingIn fact the supplied fitted cell means and equal cell sizes determine the increase without raw observations. Pooling two ten-person cells with means adds to the residual sum of squares. Here the four differences are , soTherefore and . At , the data do not reject the three-type reduction, though the result is borderline and is not proof that the pooled means are identical. This factor-level pooling test keeps the previously supported interaction structure; reducing the categories and removing all interaction simultaneously would test a different set of restrictions.
Past exam of the mathematics course of the University of Cambridge 2016 iii Paper 206 1 d Solution Created 2026-10-03 Updated 2026-10-06
In the larger normal linear model, add laboratory intercept offsets to the mean in part (a). The nested-model F-test comparesThe smaller and larger models fit four and six mean coefficients respectively. Under , independent Gaussian errors with common variance give independent quantities and . Thus the test statistic isThe p-value is the upper-tail probability . At the conventional 5% significance level, do not reject the common-intercept restriction. Prefer the simpler model
tumour1 on this comparison. This does not prove the intercepts coincide, nor address the curvature identified in part (b); the exact F-test calibration assumes the underlying error and mean assumptions. The one-decimal RSS values give when subtracted, whereas the table's difference uses unrounded values. Past exam of the mathematics course of the University of Cambridge 2017 ii Paper 1 5J i Solution Created 2026-09-24 Updated 2026-10-05
The comparison is a nested-model F-test of the three time-by-diet interaction coefficients, with null hypothesis that all are zero. The reduced linear regression model has a common time slope and four diet-specific intercepts; the full model permits four slopes. It has eight fitted coefficients and residual degrees of freedom.The rounded sums of squares give the displayed approximation; the software reports using unrounded values. The quoted p-value gives overwhelming evidence against equal log-weight growth slopes across diets under the normal linear model assumptions. Thus reject the common-slope hypothesis. Because repeated measurements on the same chicken may be correlated, the exact reference distribution requires independent, homoscedastic Gaussian errors; the displayed ordinary regression output by itself does not establish that assumption.
Past exam of the mathematics course of the University of Cambridge 2018 iii Paper 218 6 c Solution Created 2026-10-03 Updated 2026-10-05
The command performs a nested-model F-test for against , retaining the intercept and coefficient in both models. Write and for the residual sum of squares in the reduced and full fits. Under and the independent normal-error assumptions,By Cochran's theorem, the numerator and denominator are independent scaled chi-squares with one and degrees of freedom. Equivalently, for this single-coefficient restriction, , so its p-value equals that of the two-sided coefficient Student's t-test in model3.
Using the printed value gives and . The printed estimate and standard error instead giveThis discrepancy is present in the PDF and is too large to be explained by rounding at the shown precision; the underlying ANOVA output is omitted, so an exact unique statistic cannot be recovered. The univariate correlation and value imply , consistent with after rounding. With denominator degrees of freedom 97, the two versions give p-values approximately and , respectively. Either way the partial effect is not significant at 5%.
Past exam of the mathematics course of the University of Cambridge 2018 ii Paper 2 5J Solution Created 2026-09-24 Updated 2026-10-03
Let and be the residual sums of squares under the restricted model with coefficients and the full model with coefficients. Assuming has full column rank, the nested-model F-test usesUnder the null hypothesis this has the distribution.
Maximizing the Gaussian likelihood over and the unknown variance gives . Thus the likelihood-ratio test statistic isSincewe obtainThe logarithm is strictly increasing, so the two test statistics give exactly the same rejection ordering.
Past exam of the mathematics course of the University of Cambridge 2023 ii Paper 4 5J iii Solution Created 2026-09-24 Updated 2026-09-25
fit3 <- lm(log(price) ~ mpg + psngr + length + width + weight + factor(make), data = cars)The models are nested. The first nested-model F-test compares fit2 with the intercept-only fit1. Adding the five quantitative predictors reduces the residual sum of squares from to , withThere is overwhelming evidence that these quantitative car properties jointly improve the model.
The second test compares fit3 with fit2. Adding make reduces the residual sum of squares further to , withThus manufacturer has a highly significant effect even after adjusting for the five quantitative properties, and fit3 is preferred among these nested models.
There are observations because the intercept-only model has residual degrees of freedom. The make term uses additional degrees of freedom. By the degrees of freedom of a factor predictor, a factor with represented levels contributes degrees of freedom when an intercept is present. Henceunique manufacturers occur in the dataset.
Treatment coding 2026-10-06
Treatment coding sets one reference level in a regression factor to zero and represents each other level by an indicator column. With two interacting factors, a main-effect coefficient compares levels at the other factor's reference level; an interaction coefficient is a difference of these contrasts. A joint nested-model F-test can be significant even when some individual coefficient tests are not.
