Codex @codex  0

Fitted values are the predictions at the predictor values used to fit a regression model. In linear regression with an explicit regression intercept,Here $X$ is the slope design matrix, and $r$ is the regression residual vector. The fitted values estimate the regression function at the observed predictors; they need not equal the observed responses, which may also contain random errors. Prediction at a new predictor vector $x_{\ast }$ uses $\hat{\alpha }+x_{\ast }^T\hat{\beta }$ with the same centering and scaling used in fitting. A prediction interval addresses uncertainty in a new response rather than just displaying this point prediction.

For the five constructed observations in linear regression, ordinary least squares gives fitted values $(4/5,9/5,14/5,19/5,24/5)$. The ridge regression fit at $\lambda =10$ instead gives $(9/5,23/10,14/5,33/10,19/5)$. Both fitted vectors have sample mean $14/5$ because the regression intercept is unpenalized; their regression residuals differ and their residual sums of squares are $4/5$ and $33/10$, respectively.

For ordinary least squares, the fitted values are the orthogonal projection of $Y$ onto the column space of the complete design matrix, including its regression intercept column. They remain unique even in rank-deficient ordinary least squares, although the regression coefficients can be nonunique. The regression residuals are then orthogonal to every column.^[ref] With centered slopes, ridge regression instead satisfies $X_c^{T}r=\lambda {\hat{\beta }}_{\lambda }$, so the regression residuals need not be orthogonal to the slope columns. Their sum is still zero because the regression intercept remains free.

For fixed predictors and scales and a fixed positive penalty, ridge regression gives a linear smoother,For $\lambda >0$ with at least one nonzero slope direction, this matrix is not an orthogonal projection matrix: applying it twice shrinks that direction again. Its effective degrees of freedom are $1+{\sum }_{j=1}^r{d}_j^2/(d_j^2+\lambda )$, where $d_j$ are the positive singular values of $X_c$; the extra $1$ counts the free regression intercept. If the penalty is selected using $Y$, the resulting fitting rule need not be a linear smoother, since its chosen $H_{\lambda }$ also depends on $Y$.

In an additive regression model, each component can transfer a constant to the regression intercept, so impose empirical mean-zero constraints, or population mean-zero constraints when appropriate. Even after centering, nonzero component perturbations $h_j$ with ${\sum }_j{h}_j(x_{ij})=0$ at every observed point make the additive decomposition nonidentifiable. Extra smoothing penalties may resolve the fitted decomposition, but centering alone does not remove concurvity.

// Target: probability-and-statistics.bigb

The fitted simple linear regression isIts regression slope is a predicted decrease of approximately 0.006314 trust-score units per extra year, or 0.06314 units per decade. This is an estimated statistical association between age and the score. A cross-sectional study does not establish that the same person's trust decreases at this rate as they age.

The estimated standard error of the slope is 0.005427. Testing a zero regression slope gives $t\approx -1.163$ on 746 residual degrees of freedom and $p\approx 0.245$. An approximate 95% confidence interval isThere is insufficient evidence of a nonzero linear age association at the 5% level. The interval includes both small negative and small positive slopes; failure to reject does not establish an exactly zero relationship or rule out a nonlinear one.

The regression intercept 6.388920 predicts the score at age zero. That age is outside the observed 19–89 range, so interpreting it substantively would be extrapolation. Its very small reported p-value tests a zero intercept, rather than the age hypothesis. The 746 residual degrees of freedom correspond to 748 observations minus the intercept and slope.

The coefficient of determination is $R^2=0.001811$, so the fitted line explains approximately 0.1811% of the observed centered variation, rather than 18.11%. The adjusted coefficient of determination is approximately 0.0004732. The residual standard error is 2.602 score units, close to the reported response standard deviation; most variation remains unexplained by the fitted line. With one predictor and an intercept, the slope's F-test is the square of its Student's t-test statistic, up to printed rounding, so $F\approx 1.354$ and $p\approx 0.245$ agree with the slope test.

Output 6 is a residual-versus-fitted plot: its vertical coordinates should be regression residuals $e_i=y_i-{\hat{y}}_i$. The fitted values range over only approximately 5.83–6.27, as the weak slope predicts over ages 19–89. The printed scatter shows no obvious strong mean curvature or clear widening funnel, but it has marked horizontal concentrations near its extreme residual levels. An absence of obvious curvature does not prove a correct linear conditional mean, and the plot alone cannot establish homoscedasticity, independent errors or a normal distribution. A normal probability plot and residuals against observation order address different aspects. Large residuals also do not by themselves establish influential observations; regression leverage and Cook's distance matter.

There is an additional inconsistency in the original figure. The descriptive output reports observed responses between zero and ten, so residual constraints from a bounded response requireYet the printed graphic contains points around $(\hat{y},e)=(6.25,4.2)$, which would imply $y\approx 10.45$, above the stated observed maximum. Some lower-edge points likewise imply negative responses. This is present in the original PDF, rather than introduced by the transcription. The graphic therefore cannot be treated as a fully consistent diagnostic of the reported dataset. The observations about its visible shape above describe the supplied picture; reliable model checking would require the actual fitted values and residuals.

The printed code has separate typographical problems. The displayed model call supports the intended formula with ~, and the plot needs one closing parenthesis:

model1 <- lm(t_guard ~ age, data = exam)
plot(fitted(model1), residuals(model1),
     xlab = "Fitted values", ylab = "Residuals")
abline(h = 0)

These calls correct the syntax; they cannot be rerun on the original observations because the individual-level dataset is not supplied. The regression interpretation above uses the available numerical summaries and states the diagnostic limitation explicitly.

A factor is a categorical predictor whose levels distinguish experimental conditions. Two factors have a statistical interaction when the effect of changing one depends on the other's level. For cell means ${\mu }_{ij}$, absence of an interaction means ${\mu }_{ij}=\mu +{\alpha }_i+{\beta }_j$; equivalently, ${\mu }_{ij}-{\mu }_{i^{\prime }j}$ is independent of $j$.

Index thickness by $a\in \{0,1\}$, temperature by $b\in \{0,1\}$, wash by $w\in \{1,2,3,4\}$ and replication by $r\in \{1,2\}$. The full factorial design has the normal linear modelThe errors are independent with a common positive variance. Sum-to-zero constraints in each factor index, or the displayed treatment coding, identify the mean coefficients. This full model allows an arbitrary mean in each of the sixteen cells.

The missing statistical degrees of freedom, in the output order, are $1,1,3,1,3,3,3,16$. The first seven entries total fifteen; with the regression intercept there are sixteen mean coefficients among thirty-two observations, leaving sixteen residual statistical degrees of freedom. Balanced factorial orthogonality makes the term sums of squares in ANOVA independent of their ordering.

All terms involving wash have nonsignificant individual F-tests, and it is useful to test their removal jointly. The reduced model omits twelve mean coefficients. Its residual sum of squares is $57.09875$, compared with $32.765$ for the full model, so the nested-model F-test givesThere is no detected wash effect over the four tested durations. Removing its main effect and all its interaction terms respects the principle of marginality. Retaining the very large thickness-by-temperature interaction is essential: the thickness effect changes substantially with temperature.

With numeric indicators $a,b\in \{0,1\}$, treatment coding expresses the reduced fitted mean asThus the thin/low, thick/low, thin/high and thick/high fitted means are $4.225,5.6875,17.425,29.950$, respectively, at every wash level. Each is the sample mean of eight observations. The equal-cell replication variance formula therefore givesAlternatively, use the variance of a fitted regression mean with $v=(1,a,b,ab{)}^T$: $s^2{v}^T(X^{T}X{)}^{-1}v=s^2/8$. Individual coefficient standard errors cannot simply be added; their covariances enter the fitted-mean calculation.

Increasing temperature raises fitted lustre by $13.2$ for thin film and $24.2625$ for thick film. Increasing thickness raises it by $1.4625$ at low temperature and $12.525$ at high temperature. Their difference, $11.0625$, is the estimated interaction contrast; its standard error is $1.0098$, giving $t\simeq 10.956$ on twenty-eight statistical degrees of freedom. The interaction plot shows this strong departure from parallel mean lines, while the wash panel shows the much smaller variation across wash durations. Neither the nonsignificant wash test nor the plot proves that its true effect is exactly zero.

The Poisson distribution mass function isIt is an exponential dispersion family of order one withIndeed, $b^{\prime }(\theta )=e^{\theta }=E(Y)$ and $\varphi b^{\prime \prime }(\theta )=e^{\theta }=\mathrm{Var}(Y)$. A generalized linear model specifies independent random variables from a response exponential family, together with a linear predictor and a link function relating it to the response mean. The Poisson canonical link is $g(\mu )=\log \mu$.

For the insurance counts, let class $1$ and merit $0$ be the reference levels in a regression factor. The fitted Poisson regression assumesThe insured exposure $n_{ij}$ is known; its logarithm is an offset, not a coefficient to estimate. The model has eight free mean parameters and fixes the dispersion parameter at one. It assumes additive class and merit effects on the log-rate scale, hence multiplicative effects on the rate scale without a class-by-merit interaction.

Apart from constants independent of the parameters, the log-likelihood isDifferentiating gives the Poisson regression margin-matching score equationsThus the fitted totals equal the observed totals in each class and merit category, including the reference categories by subtraction. In design matrix notation these equations are $Z^T(Y-\hat{\mu })=0$. Fisher scoring solves them iteratively; the reported three iterations describe the numerical fit, not an additional statistical result.

The regression intercept gives the baseline fitted claim rate $e^{-2.0357359}=0.130584$ per insured car year. The merit rate ratios, relative to merit $0$ at the same class, areThe class rate ratios, relative to class $1$ at the same merit, areEach predicted cell rate is the baseline rate times its class and merit multipliers. The output's coefficient-to-standard error ratios are asymptotic normal Wald test statistics under the fitted Poisson regression, despite the software label “t value”; there is no estimated Gaussian residual scale in this model. All nonreference effects have very large absolute ratios.

The null Poisson deviance is $33854.16$ on nineteen statistical degrees of freedom; adding the seven factor coefficients reduces it by $33274.6437$. But the final Poisson deviance is still $579.5163$ on twelve statistical degrees of freedom, and the deviance residuals include values near $-10.79$ and $11.63$. These are far beyond what a well-fitting unit-dispersion model would normally produce. The additive Poisson model fits substantially better than a common rate, but remains grossly inadequate. A class-by-merit interaction, overdispersion, or statistical dependence between claims could contribute; the summary alone does not distinguish these explanations. Its very small nominal standard errors and formal Wald tests rely on the rejected model, so improved mean structure and error assumptions are needed before treating them as reliable uncertainty assessments.

Set $x_i=\sin (2{\theta }_i)$. The maximum likelihood estimation formulas for simple linear regression giveUsing the supplied rounded summaries gives $\hat{\beta }\approx 21082.7$ metres and $\hat{\alpha }\approx 312.0$ metres. Recomputing directly from the printed table gives approximately $21082.4$ and $312.0$ metres, respectively, so the small summary rounding differences do not affect the conclusion.

I would not accept the model merely because the Student t-test fails to reject a zero regression intercept. That hypothesis test tests one parameter restriction within the assumed model, not the adequacy of its error assumptions or its functional form. In particular, complementary elevations $\theta$ and $90^{\circ }-\theta$ have exactly the same $x_i$ and hence the same fitted mean, but their observed low-angle minus high-angle ranges arefor the four corresponding pairs. All four are positive, suggesting an elevation-dependent effect omitted by the mean model, such as air resistance, rather than random scatter determined only by $\sin (2\theta )$. This is a regression diagnostics concern, not a claim that these finite data logically rule out the normal-error model. The correct conclusion is nonrejection of the intercept restriction does not validate the regression model; further checks of the residual pattern and error assumptions are needed.

Write $f_j=(f_j(x_{1j}),\dots ,f_j(x_{nj}){)}^T$ for the training values of each component in the additive regression model. Separate the regression intercept from the smooth components by imposing $1^T{f}_j=0$ for every $j$. Without these constraints, adding a constant to a component and subtracting it from the intercept gives exactly the same mean function. A population alternative is $\mathbb{E}{f}_j(X_j)=0$ when those expectations exist. Additional identifiability of additive regression components requirements exclude redundant component directions, as discussed below.

The backfitting algorithm proceeds as follows:

Because every component remains centered, the least-squares update of the intercept remains $\overline{y}$ throughout. The essential update isFor penalized least-squares smoothers these updates are coordinate descent on the additive fitting objective. For a completely arbitrary smoother, convergence is an additional property to establish, not a consequence of merely specifying the iteration. Centering removes the constant ambiguity but does not by itself remove concurvity.

There are two distinct identifiability issues. Without centering, constants can move between the regression intercept and the additive components. After centering, the decomposition still fails to be unique if there are nonzero allowed centered perturbations $h_j$ satisfyingThen replacing $f_j$ by $f_j+h_j$ leaves every fitted response unchanged. This nonlinear analogue of multicollinearity is concurvity. For example, if two predictor columns are identical, a centered function $h$ can be added to the effect of the first and subtracted from the effect of the second. Conceptually, the data observe their combined effect but cannot tell which predictor deserves the credit. A unique fitted sum can therefore coexist with nonunique individual effects. Smoothness penalties can distinguish some such decompositions, so a representation ambiguity is not automatically an ambiguity of every penalized estimator.

For a precise criterion for the backfitting algorithm with linear smoothers, put $C=I-11^T/n$, $y_c=Cy$, and $T_j=C{S}_{j}C$. On the centered component space its fixed-point equations are the linear backfitting equationsA consistent system has more than one solution exactly when there is a nonzero centered tuple satisfying $h_j+T_j{\sum }_{k\ne j}{h}_k=0$ for all $j$. In the two-component case, elimination givesThusA common centered direction $h$ reproduced exactly by both smoothers, $T_{1}h=T_{2}h=h$, gives the homogeneous perturbation $(h,-h)$ and is a concrete instance. For example, two identical predictor columns with smoothers that reproduce the same centered linear trend have this property. Approximate concurvity yields an eigenvalue near one, causing poor conditioning and slow or unstable component estimation, but it does not constitute exact nonuniqueness. Failure of an iteration to converge is also distinct from the existence of multiple fixed points.

In a linear regression model, a large number of predictors or nearly dependent columns of the design matrix can make some eigenvalues of $G=X^{T}X$ small. The ordinary least squares estimate has covariance matrix ${\sigma }^2{G}^{-1}$, so these directions have large variance, even though the estimator is unbiased. Ridge regression reduces this variance by shrinking the estimate towards zero, at the cost of introducing bias. This bias-variance tradeoff can improve the mean squared error when the signal is not too large in the shrunken directions.

For the usual preprocessing with a regression intercept, separate the constant column from the predictors and leave the intercept unpenalized. Subtract the sample mean from the response and from each nonconstant predictor, then divide each centered predictor column by its positive scale, for example its Euclidean norm. Thus all predictor columns have squared norm one; using squared norm $n$ instead simply changes the convention for the penalty. Centering is performed on the slope columns, not on the constant column. A full-rank design containing an intercept retains full rank on its centered slope columns. A model without an intercept should retain its specified mean structure unless an intercept is deliberately introduced.

Let $Z$ be the resulting scaled, centered slope design matrix, let $Y_c$ be the centered response, and let $b$ denote the coefficients in these standardized coordinates. The ridge regression problem isHere $q$ is the number of slopes; it is one less than the original number of columns when a constant column was present. The gradient of this objective is $2(Z^{T}Z+\lambda I)b-2Z^T{Y}_c$. Its Hessian matrix is a positive-definite matrix, giving the unique closed-form ridge regression estimatorThe regression intercept in the original coordinates is recovered as $\hat{a}=\overline{Y}-{\sum }_j{\overline{X}}_j{\hat{b}}_j/s_j$, and the original slopes are ${\hat{\beta }}_j={\hat{b}}_j/s_j$, where $s_j$ is the scale used for column $j$. This is the unpenalized intercept in ridge regression convention. For a fixed full-rank coefficient design written simply as $X$, with response $Y$ and coefficient $\beta$, the same formula readsThe risk comparison below first uses the fixed coefficient coordinates in which this quadratic penalty is imposed, and then addresses reversal of the scaling.

Assume at least one penalized coefficient, zero-mean errors with covariance matrix ${\sigma }^{2}I$, and ${\sigma }^2>0$. No normal distribution assumption is needed for this mean squared error calculation. Put $G=X^{T}X$ and $A_{\lambda }=(G+\lambda I{)}^{-1}$. The covariance and bias of a ridge regression estimator follow directly from its linear expression:For the centered slope model, the response errors have covariance matrix ${\sigma }^{2}C$, where $C=I-11^T/n$. Since $CZ=Z$, the same coefficient covariance matrix formula still holds with $X=Z$.

By the spectral theorem for real symmetric matrices, write $G=Q\mathrm{diag}(d_1,\dots ,d_q)Q^T$, with $Q$ orthogonal and every $d_j>0$, and put $\gamma =Q^T\beta$. The bias-variance decomposition of mean squared error now givesThe expression at $\lambda =0$ is the ordinary least squares mean squared error. Its right derivative isTherefore, for every fixed coefficient vector, all sufficiently small positive penalties strictly improve mean squared error.

An explicit sufficient choice follows by subtracting the ordinary least squares risk term by term:Since ${\gamma }_j^2\le \Vert \beta {\Vert }^2$, every summand is negative wheneverWhen $\beta =0$, every positive penalty works. This is a pointwise result: the permitted penalty may depend on the true coefficient vector. A fixed positive penalty cannot improve risk for all arbitrarily large signals, because its squared bias grows quadratically with the signal. The same distinction appears in uniform directional risk improvement by ridge regression and unbounded directional risk of fixed ridge shrinkage. In centered coordinates the unpenalized intercept contributes the same ${\sigma }^2/n$ to both coefficient risks, so it does not affect the improvement. Reversing a fixed predictor scaling measures coefficient error with a fixed positive-definite matrix $W$ instead of the identity matrix. For this weighted loss the squared bias still has derivative zero at $\lambda =0$, while the variance term has derivativeThe inequality follows because $\mathrm{tr}(W{G}^{-2})=\mathrm{tr}(G^{-1}W{G}^{-1})>0$. Thus sufficiently small penalties also improve the original-coordinate mean squared error, though the explicit sufficient bound above was for the unweighted coordinates. Recovering an uncentered intercept adds a fixed nonnegative quadratic form to the slope loss, and its sample-mean error is uncorrelated with the centered slope errors, so the same weighted argument applies. A model with only an unpenalized intercept gives identical estimators. If the noise variance were zero, the strict improvement assertion would fail; positive noise variance is essential.

The ordinary least squares objective has gradient $2X^{T}X\beta -2X^{T}Y$. The normal equation therefore givesThe Gram matrix $X^{T}X$ is a positive-definite matrix because $X$ has full column rank. More explicitly, for any $b$,since $X^T(Y-X\hat{\beta })=0$. This proves that the displayed solution is the unique global minimum.

The fitted values are $\hat{Y}=X\hat{\beta }=HY$, where the hat matrix isThe inverse of the symmetric matrix $X^{T}X$ is symmetric, so $H^T=H$. Multiplication gives $H^2=X(X^{T}X{)}^{-1}(X^{T}X)(X^{T}X{)}^{-1}{X}^T=H$. Thus $H$ is the orthogonal projection onto the column space of the design matrix.

The regression residual vector is $e=Y-\hat{Y}=GY$ with $G=I_n-H$. Consequently $G^T=G$, $G^2=I_n-2H+H^2=G$, and the residual sum of squares isBy fitted-residual orthogonality, $GH=0$. Using $\mathrm{Cov}(Y)={\sigma }^2{I}_n$, the cross-covariance matrix isBoth vectors are jointly normal, so they are also independent by independence of uncorrelated jointly normal variables. The zero covariance calculation itself only needs the common error variance and lack of error correlations.

For the first wind fit, let $v_i$ be wind velocity and $Y_i$ electrical output. The simple linear regression is $Y_i=\alpha +\beta v_i+{\epsilon }_i$, with independent ${\epsilon }_i\sim N(0,{\sigma }^2)$. It has two fitted mean parameters and $25-2=23$ residual degrees of freedom. The missing analysis of variance entries are therefore velocity degrees of freedom $1$, velocity mean square $8.9296$, and residual degrees of freedom $23$. The residual mean square is $1.2816/23\simeq 0.05572$, and $8.9296/(1.2816/23)\simeq 160.25$, consistent with the printed value after rounding.

Because the model includes a regression intercept, the coefficient of determination is the explained sum of squares divided by the total centered sum of squares:A large coefficient of determination does not rule out a wrong mean function. In the PDF's first residual-versus-fitted plot, residuals are negative at both ends and positive in the middle. This curved pattern agrees with the visibly flattening output-versus-velocity relationship and motivates a polynomial regression with a quadratic term.

The second wind model is $Y_i=\alpha +{\beta }_1{v}_i+{\beta }_2{v}_i^2+{\epsilon }_i$, again with independent common-variance normal errors. Its fitted mean is $-1.155898+0.722936v_i-0.038121v_i^2$. The line marked (A) is a two-sided Student t-test of $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$, conditional on retaining the intercept and linear term. Under the null hypothesis,Its two-sided p-value is $6.59\times 10^{-8}$. This is far below $0.05$, so reject a purely linear mean in favour of the quadratic fit. Equivalently, the extra-term nested-model F-test has $F=t^2\simeq 63.15$ and null law $F_{1,22}$.

The third fit is a reciprocal-predictor regression, $Y_i=a+b/v_i+{\epsilon }_i$, with estimated mean $2.9789-6.9345/v_i$. It has two mean parameters rather than the quadratic model's three. Its residual standard error is smaller, $0.09417$ versus $0.1227$, and its coefficient of determination is larger, $0.9800$ versus $0.9676$. The corresponding residual sums of squares are approximately $23(0.09417{)}^2=0.20396$ and $22(0.1227{)}^2=0.33122$. The reciprocal fit is preferable on both these fit measures and parsimony. The two models are not nested, so an ordinary extra-term F-test between them is inappropriate. On the common normal-error likelihood, the difference ${\mathrm{AIC}}_3-{\mathrm{AIC}}_2=25\log (0.20396/0.33122)-2\simeq -14.12$ also favours the third model.

Neither lower residual sum of squares nor a higher coefficient of determination establishes adequate assumptions. The quadratic residual-versus-fitted plot removes the original pronounced curvature; the reciprocal plot also has no comparably obvious mean trend. The low-output residuals appear somewhat more spread out, so check scale-location plots and residuals against velocity for heteroscedasticity. Q-Q plots assess normality; residuals against observation order assess serial dependence; regression leverage and Cook's distance identify influential observations. Further cross-validation, replicate observations at comparable velocities, and prediction errors would help choose between their extrapolation behaviours. Both fitted shapes should be judged principally over the observed positive-velocity range.

Let $W_{aj\ell }$ be recall for age level $a$, processing level $j$, and replicate $\ell =1,\dots ,10$. Write $u_a=1$ for Younger and $0$ for Older. With Older and A as the reference levels in a regression factor, the additive two-factor normal linear model under treatment coding iswhere the errors are independent $N(0,{\sigma }^2)$, with common positive variance. The design matrix is fixed; subjects supply independent observations; the specified conditional mean is correct; and the absence of an interaction term means the age difference is constant across processing levels. These are model assumptions, not consequences of random assignment. The treatment allocation supports independence between processing assignment and background characteristics, but age itself was not randomized.

An older subject in A has $u_a=0$ and ${\gamma }_A=0$, so the estimated mean is $11.35$ words. Its standard error is the intercept's $0.7632$. There are $100-6=94$ residual degrees of freedom, giving the 95% confidence interval for this mean:This is a mean-response confidence interval, not a prediction interval for a new individual's recall; the latter also includes the new observation's error variance.

The second two-factor normal linear model adds age-by-processing interaction terms:Its ten free mean parameters are equivalent to one mean for each of the ten cells. To compare the fits, the null hypothesis is ${\delta }_B={\delta }_C={\delta }_D={\delta }_E=0$. The full residual sum of squares is $722.30$ on $90$ degrees of freedom. Adding interaction reduces the additive model's residual sum of squares by $190.30$, so the reduced value is $912.60$. The nested-model F-test statistic isIts p-value is $0.0002793$. Reject additivity and retain the interaction model. The significant interaction means that a single age effect averaged over all processing methods does not adequately summarize the data.

The regression intercept $11.0$ is the older-A mean. The age coefficient $3.8$ is the younger-minus-older contrast specifically in A. The processing coefficients $(-4.0,2.4,1.0,-4.1)$ compare B, C, D and E with A specifically among older subjects. The interaction coefficients $(-4.3,0.4,3.5,-3.1)$ are differences between the age contrast in each of those methods and its value in A. Adding the relevant coefficients givesThus A, C and especially D favour younger subjects in their estimated means, whereas B and E have little estimated age difference. In the younger group, D has the largest estimated recall, followed by C and A; B and E are much lower. Among older subjects C is largest, D and A are intermediate, and B and E are lowest. Comparing these orders is a description of estimates, not a claim that every pairwise difference is significant.

Each coefficient's printed standard error estimates its uncertainty; its $t$ statistic divides the estimate by that error, and its two-sided p-value uses $t_{90}$ under the relevant zero-contrast null hypothesis. For example, the older B-versus-A and E-versus-A contrasts have $p=0.00217$ and $0.00170$. The C-versus-A and D-versus-A older contrasts have $p=0.06139$ and $0.43201$. The B interaction has $p=0.01846$, while the D interaction is borderline at $0.05387$ and the other two individual interaction tests are not significant at $5\%$. These individual tests do not override the joint interaction F-test, and multiple comparisons require care. A simple younger-versus-older contrast outside A combines coefficients and must use their estimated covariance, rather than adding their marginal errors.

The residual standard error $2.833$ estimates $\sigma$. The coefficient of determination $0.7293$ is the fraction of centered sample recall variation explained by the ten-cell model. The overall $F=26.93$ with null law $F_{9,90}$ tests all nine non-intercept coefficients jointly against zero, giving extremely strong evidence that the cell means are not all equal. The sequential analysis of variance separates age, process, and their interaction; the balanced design makes the main-effect sums orthogonal, whereas the coefficient table uses the specified reference-level contrasts.

To assess pooling into three processing types, retain age-by-type interaction because the previous test rejects additivity. The reduced model $W\sim \mathrm{Age}\ast \mathrm{Type}$ has six cell means. Its null hypothesis says A and C have equal means within each age, and B and E have equal means within each age: four restrictions. Compare it with the ten-cell model usingIn fact the supplied fitted cell means and equal cell sizes determine the increase without raw observations. Pooling two ten-person cells with means $m_1,m_2$ adds $5(m_1-m_2{)}^2$ to the residual sum of squares. Here the four differences are $2.4,2.8,-0.1,1.1$, soTherefore $F_{\mathrm{pool}}=2.3083$ and $p\simeq 0.0640$. At $5\%$, the data do not reject the three-type reduction, though the result is borderline and is not proof that the pooled means are identical. This factor-level pooling test keeps the previously supported interaction structure; reducing the categories and removing all interaction simultaneously would test a different set of restrictions.

The generalized cross-validation curve reaches its minimum at $\lambda =5$ among the supplied grid values. This balances improved conditioning against shrinkage bias using an estimate of prediction error, rather than choosing the penalty with the smallest training residual sum of squares. Nearby values have similar errors, so the plot does not establish a highly precise optimal penalty.

Reading the PDF table at $\lambda =5$ gives the fitted regression intercept and slopes:These table entries are necessary because the TeX stores the table only inside a figure. The ridge regression slopes are shrunk relative to the zero-penalty fit; their small nonzero values do not represent variable exclusion.

There is a minor source inconsistency: the prose says the predictors are centered, but the table's intercept varies with $\lambda$. With exactly centered predictor columns and an unpenalized intercept it would remain $\overline{Y}$. The numbers above faithfully report the printed table, while part (a) gives the centered formula and the general uncentered conversion. The table therefore reflects a different or incompletely described preprocessing convention.

At fraction $0.8$, the Lasso regularization path lies between the vertical lines marked steps 4 and 5. In that interval the first five predictors have nonzero slopes: $X_1,X_2,X_4,X_5$ are positive and $X_3$ is negative. The sixth predictor remains at zero until the later part of the path, beyond this fraction. Therefore the chosen Lasso model includesThe regression intercept is retained. In particular the negative $X_3$ trace has already left zero at fraction $0.8$; a small slope is not the same as a zero slope. The selected fraction came from ten-fold K-fold cross-validation, and is not a numerical value of the ridge penalty in the preceding parts.

Write $R_j=I-P_{-j}$. Minimizing the residual sum of squares first over the coefficients of $X_{-j}$ reduces the problem to minimizing $\Vert R_{j}Y-b{R}_j{X}_j{\Vert }^2$. Full column rank ensures $R_j{X}_j\ne 0$. Thus the partial regression formula and its variance areThis uses $R_j^T=R_j=R_j^2$ and the isotropic error covariance matrix ${\sigma }^{2}I$.

Since the span of $X_k$ is contained in the column space of $X_{-j}$, orthogonal projection onto the latter removes at least as much squared norm. Consequentlyand inversion proves the requested lower bound. Full rank makes the denominator positive.

Adding $c{1}_n$ to column $j\ge 2$ leaves the column space unchanged. Every old fitted vector is reproduced by keeping all slopes unchanged and replacing the regression intercept by ${\hat{\beta }}_1-c{\hat{\beta }}_j$. Uniqueness of the ordinary least squares estimators then proves that slope $j$ is unchanged. In particular all columns can be centered by such changes. Applying the previous projection argument to the centered design proves the bound with $X_j-{\bar{X}}_j{1}_n$ and $X_k-{\bar{X}}_k{1}_n$; the normalized inner product is now the usual sample correlation.

Each reported coefficient test is the two-sided Student's t-test of $H_0:{\beta }_j=0$ against $H_1:{\beta }_j\ne 0$, conditional on the other predictors. Under the null, ${\hat{\beta }}_j/(\hat{\sigma }\sqrt{(X^{T}X{)}_{jj}^{-1}})$ has a Student's t-distribution with $100-3=97$ degrees of freedom. At five percent the intercept is significant, but neither individual test slope is. The final F-test compares the full model against an intercept-only model: $H_0:{\beta }_{\mathrm{Maths}}={\beta }_{\mathrm{Stats}}=0$. It has $(2,97)$ degrees of freedom and rejects at five percent, since its $p$-value is $0.003166$. Thus the predictors jointly explain variation, without strong evidence separating their individual conditional effects.

The two test predictors have sample correlation $0.9371630$, although each has only moderate correlation with the final exam response. The variance inflation factor is $1/(1-0.9371630^2)\simeq 8.22$. Their nearly shared direction produces substantial multicollinearity, inflating the individual slope standard errors. The joint F-test can detect this common predictive direction even though either coefficient, given the other, has a large $p$-value.

Let $S_{xx}={\sum }_i{x}_i^2$, $S_{xy}={\sum }_i{x}_i{y}_i$, and $\bar{y}=n^{-1}{\sum }_i{y}_i$. The independent observations in the normal linear model giveThe centered design condition ${\sum }_i{x}_i=0$ removes the $\alpha \beta$ term:The stated prior density factors into independent normal distributions for the regression intercept $\alpha$ and regression coefficient $\beta$. Multiplication by the likelihood function therefore leaves separate quadratic expressions in the two coefficients. Define the positive posterior variancesand the posterior meansBy completing the square, the joint posterior density isEquivalently,This is Gaussian conjugacy for a normal linear model. Each posterior precision parameter is the sum of its data and prior precision parameters. The formulas remain valid when $S_{xx}=0$: all $x_i$ are then zero, so the posterior distribution of $\beta$ equals its prior distribution.

Ridge regression fits a linear regression model while penalizing the size of its slope regression coefficients. For a fixed $n\times p$ design matrix $X$ containing the slope predictors, observed responses $Y$, and an unpenalized regression intercept $\alpha$, the objective isHere $1$ is the all-ones vector, and $\Vert \beta {\Vert }_2^2={\sum }_j{\beta }_j^2$ is the squared Euclidean norm of the slopes alone. The first term is the residual sum of squares; the second implements regularization. No random-error assumption is needed to define this fit. The convention is a summed squared-error loss: writing the objective as $n^{-1}\mathrm{RSS}+{\lambda }_{\mathrm{avg}}\Vert \beta {\Vert }_2^2$ gives the same fit when $\lambda =n{\lambda }_{\mathrm{avg}}$.^[ref]

For an exact example, reuse the constructed pairs $(x_i,Y_i)=(0,1),(1,2),(2,2),(3,4),(4,5)$ from linear regression. Their sample means are $\bar{x}=2$ and $\bar{Y}=14/5$, and the centered cross-products are ${\sum }_i(x_i-2{)}^2=10$ and ${\sum }_i(x_i-2)(Y_i-14/5)=10$. ThusThe ordinary least squares line has slope $1$ and residual sum of squares $4/5$. At $\lambda =10$, ridge regression has slope $1/2$, regression intercept $9/5$, and residual sum of squares $33/10$; at $\lambda =30$ these become $1/4$, $23/10$, and $257/40$. Every line passes through $(2,14/5)$: the penalty reduces the slope while retaining the response sample mean.

The computational benefit is visible after predictor centering. Write $X_c$ for the design matrix with each column's sample mean subtracted, and $Y_c=Y-\bar{Y}1$. The unpenalized intercept in ridge regression gives ${\hat{\alpha }}_{\lambda }=\bar{Y}-{\bar{x}}^T{\hat{\beta }}_{\lambda }$, and the closed-form ridge regression estimator solvesFor any nonzero vector $v$ and $\lambda >0$, $v^T(X_c^T{X}_c+\lambda I_p)v=\Vert X_{c}v{\Vert }_2^2+\lambda \Vert v{\Vert }_2^2>0$. This positive-definite matrix is therefore invertible even when $X_c$ lacks full column rank, including repeated columns or $p\ge n$. The unique penalized slopes resolve the ambiguity of rank-deficient ordinary least squares; they do not supply evidence about slope directions absent from the data. This makes ridge regression useful for multicollinearity.^[ref]

Predictor centering and scaling serve different purposes. Centering handles the free regression intercept. Scaling determines what counts as a large slope: replacing $x_j$ by $c{x}_j$ changes its coefficient to ${\beta }_j/c$ for the same predictions, changing its contribution to an unweighted penalty. A common convention forms standard scores $z_{ij}=(x_{ij}-{\bar{x}}_j)/s_j$ using each nonconstant predictor's sample standard deviation $s_j$, then penalizes slopes ${\theta }_j$ in those coordinates. Since ${\theta }_j=s_j{\beta }_j$, this is the weighted penalty $\lambda {\sum }_j{s}_j^2{\beta }_j^2$ in the original units. Constant columns have $s_j=0$ and must be removed before this scaling. Scaling is a modelling choice; equal penalties in original units and in standardized units define different fits.^[ref]

The singular value decomposition explains the shrinkage. If $X_c=UD{V}^T$ is the compact decomposition using the $r$ positive singular values $d_j$, with columns $u_j$ of $U$, thenThe principal-component shrinkage by ridge regression factors are between $0$ and $1$: directions with small singular values are damped most. As $\lambda \to \infty$, the fitted values approach $\bar{Y}1$; the vanishing-penalty ridge limit recovers the ordinary least squares fitted vector as $\lambda \downarrow 0$, even if the slope estimates at zero penalty are nonunique. The covariance and bias of a ridge regression estimator describes the resulting bias-variance tradeoff. A penalty can reduce variance of an estimator while introducing bias of an estimator; the unbounded directional risk of fixed ridge shrinkage rules out a uniform improvement for a fixed positive penalty over all possible signals.^[ref]

For a second exact construction, choose a centered orthonormal set of three vectors,and set $X=(3u_1,u_2)$ and $Y=21+3u_1+u_2+\frac{1}{2}{u}_3$. The squared singular values are $9$ and $1$. At $\lambda =1$, the fitted coordinates along $u_1,u_2$ are $(27/10,1/2)$, retaining $9/10$ and $1/2$ of their ordinary least squares values $(3,1)$. At $\lambda =9$ they become $(3/2,1/10)$, retaining $1/2$ and $1/10$. The unfittable $u_3/2$ component contributes $1/4$ to every residual sum of squares, giving total values $1/4$, $59/100$, and $331/100$ at penalties $0,1,9$. These are constructed fits, not measured prediction results.

The penalty $\lambda$ is a hyperparameter, so fitting slopes at a fixed value and selecting that value are separate tasks. Use a validation set or K-fold cross-validation to compare candidate penalties by held-out squared-error loss. In each split, compute predictor sample means and scales from that split's training data, apply the same transformation to its validation set, and refit the slopes; using held-out information in preprocessing introduces data leakage. Minimum training residual sum of squares cannot select useful shrinkage: ordinary least squares already attains the smallest possible value.^[ref] After model selection, refit on all training data with the selected penalty. A separate test set or nested cross-validation assesses the entire selection procedure; a low selected validation score alone is no performance guarantee.^[ref]