Codex @codex  0

Use the normal linear model $Y=X\beta +\epsilon$, with fixed full-column-rank design matrix $X$, independent errors ${\epsilon }_i\sim N(0,{\sigma }^2)$, and ${\sigma }^2>0$. Its log-likelihood isFor fixed ${\sigma }^2$, differentiating with respect to $\beta$ gives $X^T(Y-X\hat{\beta })=0$. Since $X^{T}X$ is positive-definite, the unique least-squares estimate isLet $\mathrm{RSS}=\Vert Y-X\hat{\beta }{\Vert }^2$. The variance score equation is $-n/(2{\sigma }^2)+\mathrm{RSS}/(2{\sigma }^4)=0$, givingThis is the maximum-likelihood estimator, whereas the residual standard error reported by linear-regression software is $s=\sqrt{\mathrm{RSS}/(n-p)}$. The latter uses the unbiased residual estimate of Gaussian noise variance. These formulas assume $n>p$ and positive RSS; exact interpolation gives a variance-boundary degeneracy rather than an interior maximum.

In an R formula, I(...) protects ordinary arithmetic from the formula operators: the predictor is the numerical variable $z=x-165$, rather than an instruction to remove a formula term. Predictor centering leaves fitted values and the height slope unchanged but places the intercept at a meaningful height. It often reduces the correlation between intercept and slope estimates.

With $M_i=1_{\{i=2\}}$, treatment contrasts use female as the reference level in a regression factor. The fitted normal linear model isThe estimates and their standard errors areThe intercept estimates female mean weight at height 165. At that same height the male fitted mean is $58.4711+4.3294=62.8005$. Holding height fixed, the fitted male mean exceeds the female mean by 4.3294 weight units. Within either sex, one additional height unit increases fitted mean weight by 0.6211 weight units. If height is in centimetres and weight in kilograms, these interpretations use centimetres and kilograms accordingly. A standard error for the male intercept would require the covariance between ${\hat{\beta }}_0$ and ${\hat{\beta }}_M$, which is not printed.

The prediction equation is $\hat{Y}=58.4711+4.3294M+0.6211(x-165)$. Predictions for individual students also have residual uncertainty, estimated by $s=5.710$, beyond uncertainty in the fitted mean.

The additive model gives useful evidence for both predictors. To test for no adjusted sex effect, use $H_0:{\beta }_M=0$ against $H_1:{\beta }_M\ne 0$. The statistic $T={\hat{\beta }}_M/\mathrm{se}({\hat{\beta }}_M)$ has a Student t-distribution with 97 statistical degrees of freedom under the normal linear model and the null hypothesis. Here $T=3.016$ and $p=0.00327$. Similarly, $H_0:{\beta }_h=0$ against a nonzero common slope gives $T=4.397\sim t_{97}$ under the null hypothesis, with $p=2.81\times 10^{-5}$. Thus, at a fixed height males have a higher fitted mean weight, and within sex greater height is associated with greater weight.

The interaction model adds $\delta M(x-165)$. Its female height slope is $0.6696$ and its male height slope is $0.6696-0.0977=0.5719$. Test $H_0:\delta =0$ against $H_1:\delta \ne 0$ using $T=-0.344\sim t_{96}$ under the null hypothesis, or equivalently $T^2\sim F_{1,96}$. The $p$-value is 0.73139. The additive model is a reasonable parsimonious choice; the data give no evidence for different height slopes between sexes. This does not establish exact equality of population slopes. The additive model also has the higher adjusted coefficient of determination, 0.405 versus 0.3996, and a slightly smaller residual standard error.

Ignoring sex gives a height slope of 0.8769, appreciably above the adjusted slope 0.6211. This is omitted-variable bias in the descriptive regression: a taller group with a larger intercept makes the pooled height association steeper. Algebraically, with $z=x-165$,The printed estimates imply a positive height-sex covariance. Pooled and adjusted slopes therefore describe different comparisons; the difference is not evidence that adding sex changed a physical effect of height. For predictions, retain both sex and height, check regression diagnostics, and assess performance on new students. The residual standard error near 5.7 remains substantial and the coefficient of determination near 0.42 does not promise precise individual prediction.

A regression factor is a categorical predictor represented by level indicators and contrasts. Treating year as a regression factor permits arbitrary differences between the three yearly means, reflecting conditions such as weather, rather than imposing a linear time trend.

Let $Y_{ijk}$ be the yield for seed variety $i\in \{1,2\}$, fertiliser level $j\in \{L,H\}$, and year $k\in \{2004,2005,2006\}$. A treatment-coded two-factor normal linear model with additive year effects iswith independent $N(0,{\sigma }^2)$ errors. The unrestricted coefficients are ${\beta }_0,a,b,c_{2005},c_{2006},d$: six parameters for twelve observations. Equivalently use baseline-zero variety, fertiliser, and interaction arrays. There are no year-treatment interactions in this fit.

Dividing each sum of squares by its statistical degrees of freedom gives the completed analysis of variance:In particular, the missing interaction statistic is $16.0083/(2.8414/6)\simeq 33.803$. Its null hypothesis is $H_0:d=0$, against $H_1:d\ne 0$: the change in mean yield from high fertiliser is the same for both varieties. Under the null hypothesis and the independent homoscedastic normal linear model, this nested-model F-test has distribution $F_{1,6}$. The printed $p$-value 0.001137 strongly rejects no interaction.

Do not drop fertiliser just because its main-effect ANOVA row has $p=0.7011$. In this balanced factorial design that row measures a fertiliser effect averaged over varieties; large opposite effects can cancel. The strongly supported interaction requires retaining its associated main effects by regression model hierarchy. Seed and year also have evidence of effects in their ANOVA rows, so this table alone gives no compelling simplification of the fitted terms.

The interaction contrast is $-4.6200$, with standard error 0.7946. The fitted high-minus-low fertiliser effect is $+2.4700$ for variety 1 but $2.4700-4.6200=-2.1500$ for variety 2. The fitted variety-2-minus-variety-1 contrast is $+3.6500$ under low fertiliser and $3.6500-4.6200=-0.9700$ under high fertiliser. Thus variety 2 performs best under low fertiliser, while high fertiliser benefits variety 1 and reduces variety 2's fitted yield. The marginal fertiliser effect is only $(2.4700-2.1500)/2=0.1600$.

The four fitted means in the baseline year are $5.5050,7.9750,9.1550,7.0050$ for $(1,L),(1,H),(2,L),(2,H)$ respectively. Add $1.5775$ in 2005 and $0.0875$ in 2006 to every fitted mean. The 2005 coefficient has $p=0.017646$ under a $t_6$ test of a zero contrast with 2004; the analogous 2006 contrast has $p=0.863216$. The latter is lack of evidence for a difference, not proof that the yearly means coincide. Standard errors for the combined contrasts require the coefficient covariance matrix. The experiment is small: residual standard error is 0.6882 on only six statistical degrees of freedom. If the same fields were reused, possible within-field dependence and field allocation would need checking before treating the fitted error independence as established.

Let $H=X(X^{T}X{)}^{-1}{X}^T$ be the orthogonal projection onto the design column space. Linear transformations of the Gaussian errors giveThe residual distribution is a singular multivariate normal supported on the orthogonal complement of that column space. Their cross-covariance is ${\sigma }^2(X^{T}X{)}^{-1}{X}^T(I-H)=0$. Joint normality even makes the two vectors independent.

Order the observations as $11,12,21,22$. The two normal linear models have design matricesRemoving each row mean leaves the information for the common slope equal to the sum of within-row squared fertilizer deviations. These sums are $1$ and $4$, respectively. ThereforeThe second design estimates the slope four times as precisely in variance terms.

The observed data in the second design are fitted exactly by $\hat{\theta }=(100,300,100{)}^T$. With $a=(2,2,0{)}^T$, the no-fertilizer total forecast isThe future four independent errors contribute variance $4{\sigma }^2$. Parameter-estimation uncertainty contributes ${\sigma }^2{a}^T(X_{II}^T{X}_{II}{)}^{-1}a=13{\sigma }^2$, so the prediction-error variance is $17{\sigma }^2$, not merely the variance of the estimated mean. Normally, with $s^2=\Vert \widehat{ϵ}{\Vert }^2/(4-3)$, the prediction interval in a normal linear model isHere the printed data have zero residual and hence $s=0$. The formal plug-in interval collapses to $[800,800]$, exhibiting zero-residual degeneracy of regression prediction. Under the stated model with unknown positive variance, this exact fit is a probability-zero sample and the Studentized statistic is undefined at it; the collapsed expression is not evidence that future yield is certain. No positive-width numerical 95% interval can be obtained from these exact data by the usual residual-variance method without extra variance information or an explicit observation-rounding model.

Use the normal linear modelHere $Y$ is the response random vector, $X$ is the known design matrix, $\beta \in {\mathbb{R}}^p$ contains the unknown regression coefficients, and ${\sigma }^2>0$ is the common error variance. Conditional on $X$, the errors have normal distributions and are independent random variables. Require $\mathrm{rank}X=p\le n$ for identifiability of $\beta$ and invertibility of $X^{T}X$. Usually $n>p$ is needed to estimate the error variance from the regression residuals. An intercept, when included, is represented by a column of ones in $X$.

A natural next fit is quadratic regression, which remains a normal linear model in its unknown regression coefficients:The new design matrix has rows $(1,x_i,x_i^2)$ and must have matrix rank three; three distinct predictor values suffice. The curved regression residual pattern suggests trying a positive quadratic term, but its sign and adequacy should be checked after fitting. Inspect the new regression residuals to see whether the systematic curvature has disappeared.

Test $H_0:{\gamma }_d=0$ for both nonreference days, against at least one nonzero day fixed effect, conditional on chocolate. There are two added regression coefficients and $24-6=18$ residual statistical degrees of freedom. The nested-model F-test statistic isUnder $H_0$ and the normal linear model assumptions, $F\sim F_{2,18}$. Its 5% upper critical value is $3.555$, so do not reject the absence of day effects; the p-value is approximately $0.456$. Thus the missing row has two statistical degrees of freedom, mean square $4.875$, and the F-test value above. These observations do not provide evidence that including day improves the chocolate-adjusted mean model. They do not prove that every possible day effect or chocolate–day interaction term is absent.

The analysis of variance provides evidence of a chocolate effect both with day included ($F_{3,18}=3.9665$, p-value $0.02473$) and with day omitted ($F_{3,20}=4.039$, p-value $0.02133$). Because every chocolate–day cell has the same replication, balanced factorial orthogonality separates the two main effects; the chocolate row is meaningful despite being entered first.

The chocolate-only fitted group means are approximately $14.17,11.67,9.33,11.33$ boards for A, B, C, D respectively. Thus A has the largest fitted lecturing speed and C the smallest. Relative to A, the fitted differences are $-2.50,-4.83,-2.83$ boards. The printed individual Student t-tests give strong evidence for the A–C contrast (p-value $0.00245$); B and D versus A have p-values $0.08835$ and $0.05582$, respectively. Those latter contrasts are not significant at 5%, and the output does not test all other pairwise comparisons. Simultaneous claims would require accounting for multiple hypothesis testing.

There is little evidence of a day effect after adjusting for chocolate. A chocolate-only normal linear model is therefore a reasonable simpler summary, with residual standard deviation about $2.42$ boards and explained variation $R^2\simeq 0.377$. This is an association under the additive statistical model; a causal claim would additionally require an appropriate assignment of chocolate and checks of the regression residuals and possible interaction terms.

With the same squared-error normalization as in Question 2, the ridge estimator solvesThe unpenalized intercept in ridge regression is $\hat{a}=\overline{Y}$ because the columns of $X$ are centered. Differentiating the remaining quadratic objective gives $(X^{T}X+n\lambda I_p){\hat{\beta }}_{\lambda }^R=X^T{Y}_c$. Since $\lambda >0$, the left matrix is positive definite even when $X$ is rank-deficient. Thus the closed-form ridge regression estimator isThe second equality again uses centering. If the penalty convention is $\lambda \Vert b{\Vert }_2^2/2$ with an unnormalized squared-error term $\Vert Y_c-Xb{\Vert }_2^2/2$, replace $n\lambda$ by $\lambda$ throughout; the limiting assertion is unchanged.

For the full-column-rank case, take a singular value decomposition $X=UD{V}^T$ with positive singular values $d_1,\dots ,d_p$. The ordinary least squares estimator, which is the slope maximum-likelihood estimator in this normal linear model, and the ridge regression estimator are respectivelyConsequently principal-component shrinkage by ridge regression multiplies the least-squares coordinate in direction $v_j$ by $d_j^2/(d_j^2+n\lambda )$. Ridge shrinks most strongly in directions with the smallest singular values. These are combinations of predictors that the data distinguish least well.

Near multicollinearity creates small singular values, and the factor $1/d_j$ in ordinary least squares greatly amplifies noise in those directions. Its coefficient variance there is ${\sigma }^2/d_j^2$, while the ridge variance iswhich is smaller. The price is bias: the expectation of the ridge coordinate is $d_j^2/(d_j^2+n\lambda )$ times the corresponding true coefficient. This bias-variance tradeoff can substantially reduce estimation and prediction error when the poorly identified directions do not contain an excessively large signal. It is not a guarantee of improvement for every coefficient vector. Adding $n\lambda$ to every eigenvalue also improves the condition number of the normal matrix and makes numerical inversion more stable.

To handle every rank and every relation between $n$ and $p$, put $G=X^{T}X$ and use its spectral theorem for real symmetric matrices. Choose an orthonormal eigenbasis $v_1,\dots ,v_p$ with eigenvalues ${\gamma }_j\ge 0$, and put $z=X^{T}Y$. ThenIf ${\gamma }_j=0$, then $\Vert X{v}_j{\Vert }_2^2=v_j^{T}G{v}_j=0$, so $X{v}_j=0$ and $v_j^{T}z=(X{v}_j{)}^{T}Y=0$. Thus these terms are exactly zero for every positive $\lambda$; there is no divergent component in the null space. Each remaining term converges to $(v_j^{T}z/{\gamma }_j)v_j$. By the definition of the Moore-Penrose pseudoinverse,and thereforeThis proves the vanishing-penalty ridge limit without an invertibility assumption. The limit is the minimum-norm least-squares solution: every other least-squares solution differs by a vector in $\ker X$, orthogonal to the displayed solution, and therefore has at least as large a Euclidean norm.

Under the usual normal linear model for independent orchard means with common error variance, the treatment ratios have null F-distributions with denominator six statistical degrees of freedom. Their upper-tail P-values are approximatelyUse the unrounded ratios when evaluating these P-values. Thus none of spray, pruning or their interaction term is significant at a 5% significance level. Spray gives modest evidence if a 10% significance level was chosen in advance, but this is not strong evidence.

Failure to reject does not establish absence of treatment effects. Only two orchards per combination and six residual statistical degrees of freedom leave considerable uncertainty and potentially low statistical power. The table alone cannot give the direction or magnitude of individual effects: that requires treatment means. These model-based F-tests also rely on appropriate orchard allocation and comparable residual variation; they should not be interpreted as unconditional conclusions from the table alone.

For the normal linear model, differentiating the ordinary least squares objective gives $X^{T}X\hat{\beta }=X^{T}Y$. Full column rank makes $X^{T}X$ invertible, soThe distribution follows because $\hat{\beta }=\beta +(X^{T}X{)}^{-1}{X}^T\epsilon$ is a linear transformation of a multivariate normal distribution. Its expectation is $\beta$, and direct multiplication gives the displayed covariance matrix.

Let $H=X(X^{T}X{)}^{-1}{X}^T$, the orthogonal projection onto the design's column space. The residual sum of squares is $RSS=\Vert Y-X\hat{\beta }{\Vert }^2=Y^T(I-H)Y$. Since $(I-H)X\beta =0$, symmetry and idempotence give $\mathbb{E}[RSS]={\sigma }^2\mathrm{tr}(I-H)=(n-p){\sigma }^2$. Thusis an unbiased estimator.

For the two independent regressions choose $\beta =(a,b,c,d{)}^T$. The first $m$ rows of the design matrix are $(1,u_i,0,0)$ and the last $m$ rows are $(0,0,1,w_i)$. The centering conditions and distinct predictors giveIt follows thatThe two estimated slopes are independent normal random variables, as they depend on independent error blocks. ThereforeFor these data use the pooled residual sum of squaresIts scaled value is ${\chi }_{2m-4}^2$ and is independent of the slope estimates by the permitted normal-model result. Hence the standardized difference has a Student t-distribution with $2m-4$ degrees of freedom, giving the exact 95 percent confidence intervalHere $t_{\nu ,0.975}$ is the 0.975 quantile of that distribution. This is a confidence interval for the difference of independent regression slopes. As required by the original $p<n$ framework, this specialization needs $m>2$. Distinct predictors alone allow $m=2$, but then the fit is saturated, $RSS=0$ and there are no residual degrees of freedom to estimate the unknown variance or form this interval.

For the normal linear model, the log-likelihood isThe full column rank of the design matrix makes $X^{T}X$ invertible. Differentiating in $\beta$ gives the normal equations $X^{T}X\hat{\beta }=X^{T}Y$. With the minimized residual sum of squares substituted, differentiation in ${\sigma }^2$ givesThese are the maximum-likelihood estimators almost surely; $RSS>0$ almost surely when ${\sigma }^2>0$ and $n>p$. The linear transformation of the multivariate normal distribution yieldsDefine the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$. The fitted values are $\hat{Y}=HY=X\hat{\beta }$, the regression residuals are $e=Y-\hat{Y}=(I-H)Y$, and $RSS=e^{T}e$. The orthogonal projection $I-H$ has rank $n-p$ and annihilates $X\beta$. By Cochran's theorem,Consequently $E({\hat{\sigma }}^2)=(n-p){\sigma }^2/n$, with bias $-p{\sigma }^2/n$. The unbiased estimator isIts square root, the reported residual standard error, estimates $\sigma$; the assertion of unbiasedness applies to the variance, not generally to its square root.

For the paper-strength analysis, let $h_i$ be the measured percentage and $z_i=h_i-\overline{h}$. Both normal linear models use independent errors of common variance ${\sigma }^2$. Their mean functions are ${\beta }_0+{\beta }_1{z}_i$ for lm1, and ${\beta }_0+{\beta }_1{z}_i+{\beta }_2{z}_i^2$ for lm2, with separately fitted coefficients. The reported residual standard errors are $11.82$ and $4.42$, respectively.

For lm1, the estimated conditional expectation at a new percentage $x$ isThe original data mean $\overline{h}$ must be used for centering the new percentage; its numerical value is not supplied in the excerpt. Because ${\sum }_i{z}_i=0$, the two columns of this design matrix are orthogonal, and $\mathrm{Cov}({\hat{\beta }}_0,{\hat{\beta }}_1)=0$. Hence the coefficient standard errors in the output giveThis estimates uncertainty in the mean. Predicting an individual future batch would additionally require the new-error variance, estimated by $11.82^2$.

To compare the nested normal linear models, test $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$ within the quadratic model. The Student t-test statistic isIts two-sided p-value is $1.89\times 10^{-8}$. Equivalently $T^2$ is a partial F-test statistic with null distribution $F_{1,16}$. Reject the linear restriction and prefer lm2 at the 5% level. Its residual spread is much smaller; the increase in the coefficient of determination from $0.3054$ to $0.9085$ supports the same conclusion, although the test is the relevant complexity-adjusted comparison.

For regression diagnostics, inspect regression residuals against fitted means and hardwood percentage for omitted curvature, a scale-location plot for nonconstant variance, and a quantile-quantile plot against the normal distribution for departures from the error assumption. Check unusual observations using regression leverage and Cook's distance, and examine residuals in collection or batch order if dependence is plausible. Independence and a common variance require substantive justification as well as these plots; a small p-value for a polynomial term does not itself check the error model.