Past exam of the mathematics course of the University of Cambridge / 2012 / iii / Paper 37 / 1

Use the normal linear model $Y=X\beta +\epsilon$, with fixed full-column-rank design matrix $X$, independent errors ${\epsilon }_i\sim N(0,{\sigma }^2)$, and ${\sigma }^2>0$. Its log-likelihood isFor fixed ${\sigma }^2$, differentiating with respect to $\beta$ gives $X^T(Y-X\hat{\beta })=0$. Since $X^{T}X$ is positive-definite, the unique least-squares estimate isLet $\mathrm{RSS}=\Vert Y-X\hat{\beta }{\Vert }^2$. The variance score equation is $-n/(2{\sigma }^2)+\mathrm{RSS}/(2{\sigma }^4)=0$, givingThis is the maximum-likelihood estimator, whereas the residual standard error reported by linear-regression software is $s=\sqrt{\mathrm{RSS}/(n-p)}$. The latter uses the unbiased residual estimate of Gaussian noise variance. These formulas assume $n>p$ and positive RSS; exact interpolation gives a variance-boundary degeneracy rather than an interior maximum.

The regression residual vector is $e=Y-X\hat{\beta }=(I-H)Y$, where $H=X(X^{T}X{)}^{-1}{X}^T$ is the hat matrix. Both $H$ and $I-H$ are symmetric idempotent matrices, and $(I-H)X=0$. Therefore $e=(I-H)\epsilon$ and the linear image of a multivariate normal vector givesThis multivariate normal distribution is singular, with rank $n-p$: fitted-residual orthogonality forces $X^{T}e=0$. Specifically, $\mathrm{Var}(e_i)={\sigma }^2(1-h_{ii})$ and $\mathrm{Cov}(e_i,e_j)=-{\sigma }^2{h}_{ij}$ for distinct observations. Thus regression residuals are generally neither independent nor identically distributed, even though the model errors are. Also $\mathrm{RSS}/{\sigma }^2\sim {\chi }_{n-p}^2$ and is independent of $\hat{\beta }$, which justifies the standard errors and exact Student t-tests below.

In an R formula, I(...) protects ordinary arithmetic from the formula operators: the predictor is the numerical variable $z=x-165$, rather than an instruction to remove a formula term. Predictor centering leaves fitted values and the height slope unchanged but places the intercept at a meaningful height. It often reduces the correlation between intercept and slope estimates.

With $M_i=1_{\{i=2\}}$, treatment contrasts use female as the reference level in a regression factor. The fitted normal linear model isThe estimates and their standard errors areThe intercept estimates female mean weight at height 165. At that same height the male fitted mean is $58.4711+4.3294=62.8005$. Holding height fixed, the fitted male mean exceeds the female mean by 4.3294 weight units. Within either sex, one additional height unit increases fitted mean weight by 0.6211 weight units. If height is in centimetres and weight in kilograms, these interpretations use centimetres and kilograms accordingly. A standard error for the male intercept would require the covariance between ${\hat{\beta }}_0$ and ${\hat{\beta }}_M$, which is not printed.

The prediction equation is $\hat{Y}=58.4711+4.3294M+0.6211(x-165)$. Predictions for individual students also have residual uncertainty, estimated by $s=5.710$, beyond uncertainty in the fitted mean.

The additive model gives useful evidence for both predictors. To test for no adjusted sex effect, use $H_0:{\beta }_M=0$ against $H_1:{\beta }_M\ne 0$. The statistic $T={\hat{\beta }}_M/\mathrm{se}({\hat{\beta }}_M)$ has a Student t-distribution with 97 statistical degrees of freedom under the normal linear model and the null hypothesis. Here $T=3.016$ and $p=0.00327$. Similarly, $H_0:{\beta }_h=0$ against a nonzero common slope gives $T=4.397\sim t_{97}$ under the null hypothesis, with $p=2.81\times 10^{-5}$. Thus, at a fixed height males have a higher fitted mean weight, and within sex greater height is associated with greater weight.

The interaction model adds $\delta M(x-165)$. Its female height slope is $0.6696$ and its male height slope is $0.6696-0.0977=0.5719$. Test $H_0:\delta =0$ against $H_1:\delta \ne 0$ using $T=-0.344\sim t_{96}$ under the null hypothesis, or equivalently $T^2\sim F_{1,96}$. The $p$-value is 0.73139. The additive model is a reasonable parsimonious choice; the data give no evidence for different height slopes between sexes. This does not establish exact equality of population slopes. The additive model also has the higher adjusted coefficient of determination, 0.405 versus 0.3996, and a slightly smaller residual standard error.

Ignoring sex gives a height slope of 0.8769, appreciably above the adjusted slope 0.6211. This is omitted-variable bias in the descriptive regression: a taller group with a larger intercept makes the pooled height association steeper. Algebraically, with $z=x-165$,The printed estimates imply a positive height-sex covariance. Pooled and adjusted slopes therefore describe different comparisons; the difference is not evidence that adding sex changed a physical effect of height. For predictions, retain both sex and height, check regression diagnostics, and assess performance on new students. The residual standard error near 5.7 remains substantial and the coefficient of determination near 0.42 does not promise precise individual prediction.

The largest positive regression residual divided by the residual standard error is about $26.6071/5.942=4.48$, $24.7151/5.710=4.33$, and $24.5908/5.737=4.29$ in the three models. In comparison, the most negative regression residual is only about two residual standard errors below zero. At least one observation is much heavier than its fitted value, even after adjusting for sex and height.

A standardized regression residual is $e_i/[s\sqrt{1-h_{ii}}]$, so the regression leverage adjustment can make these ratios larger. The maximum is selected among 100 correlated regression residuals, however; it is not a prespecified single standard-normal observation. These summaries do not give a formal regression outlier test or identify which student is extreme. Ask to check the measurement and data entry, examine residual plots and a normal quantile-quantile plot, and inspect regression leverage and influential observations. A valid observation might reveal a heavier-tailed error distribution, an omitted predictor, or a nonlinear relation. Do not remove it solely because it is unusual, and do not infer from the three summaries alone that all three maxima belong to the same student.