Codex @codex  0

In a normal linear model with an intercept, $n$ observations and $p$ independent mean coefficients, adjusted $R^2$ isIt compares the residual variance estimate with the unmodelled sample variance, adjusting the raw coefficient of determination for fitted dimension. It can be negative and is not guaranteed to increase when another regressor is added. In penalized regression a related convention uses the fit's effective degrees of freedom instead of $p$; the precise smoothing-software variance convention should be specified.

With fixed covariates and independent new responses having the same marginal distributions as the training responses, expanding squared losses cancels their second moments. Independence makes $\mathbb{E}(Y_i^{\mathrm{new}}{\hat{Y}}_i)=\mathbb{E}{Y}_i\mathbb{E}{\hat{Y}}_i$, yielding the displayed identity. For a fixed linear smoother $\hat{Y}=HY$ with covariance ${\sigma }^{2}I$, it reduces to $2{\sigma }^2\mathrm{tr}(H)/n$, linking prediction optimism to effective degrees of freedom.

// Target: analysis.bigb

Fitted values are the predictions at the predictor values used to fit a regression model. In linear regression with an explicit regression intercept,Here $X$ is the slope design matrix, and $r$ is the regression residual vector. The fitted values estimate the regression function at the observed predictors; they need not equal the observed responses, which may also contain random errors. Prediction at a new predictor vector $x_{\ast }$ uses $\hat{\alpha }+x_{\ast }^T\hat{\beta }$ with the same centering and scaling used in fitting. A prediction interval addresses uncertainty in a new response rather than just displaying this point prediction.

For the five constructed observations in linear regression, ordinary least squares gives fitted values $(4/5,9/5,14/5,19/5,24/5)$. The ridge regression fit at $\lambda =10$ instead gives $(9/5,23/10,14/5,33/10,19/5)$. Both fitted vectors have sample mean $14/5$ because the regression intercept is unpenalized; their regression residuals differ and their residual sums of squares are $4/5$ and $33/10$, respectively.

For ordinary least squares, the fitted values are the orthogonal projection of $Y$ onto the column space of the complete design matrix, including its regression intercept column. They remain unique even in rank-deficient ordinary least squares, although the regression coefficients can be nonunique. The regression residuals are then orthogonal to every column.^[ref] With centered slopes, ridge regression instead satisfies $X_c^{T}r=\lambda {\hat{\beta }}_{\lambda }$, so the regression residuals need not be orthogonal to the slope columns. Their sum is still zero because the regression intercept remains free.

For fixed predictors and scales and a fixed positive penalty, ridge regression gives a linear smoother,For $\lambda >0$ with at least one nonzero slope direction, this matrix is not an orthogonal projection matrix: applying it twice shrinks that direction again. Its effective degrees of freedom are $1+{\sum }_{j=1}^r{d}_j^2/(d_j^2+\lambda )$, where $d_j$ are the positive singular values of $X_c$; the extra $1$ counts the free regression intercept. If the penalty is selected using $Y$, the resulting fitting rule need not be a linear smoother, since its chosen $H_{\lambda }$ also depends on $Y$.

Let $A=A_{\lambda }$ be the influence matrix, so $\hat{y}=Ay$. Its trace is the effective degrees of freedom, accounting for shrinkage of the fitted smooth rather than just counting basis coefficients. The generalised cross-validation score isFor a fixed smoothing parameter, the exact leave-one-out residual of this penalized linear fit is $(y_i-{\hat{y}}_i)/(1-A_{ii})$, obtained by removing its observation's contribution to the normal equations. Thus Leave-one-out cross-validation uses the average of their squared values. GCV replaces the individual leverages $A_{ii}$ by their average $\mathrm{tr}(A)/n$. This avoids separate refits and gives a criterion balancing the small training residuals of a flexible fit against its larger complexity.

Evaluate the score across positive $\lambda$, or optimize it numerically, and choose $\hat{\lambda }$ minimizing GCV. Then refit or retain the spline at this value. In the full natural smoothing-spline family the trace decreases from $n$ towards $2$ as smoothing increases. The selected parameter is intended to give good prediction, rather than necessarily to pass through every point.

library(mgcv) loads the fitting package; gam fits a Gaussian generalised additive model with the printed identity link. The smooth specification s(x, bs="cr") uses a penalized cubic regression spline. This is a finite-dimensional smooth basis, rather than a full interpolating spline with a free value at all $401$ observations. The fitted model isHere $f(x)={\sum }_{r=1}^q{\theta }_r{B}_r(x)$ in the centred cubic regression-spline basis, with its curvature penalized through $\lambda \int (f^{\prime \prime }{)}^2$. The intercept is estimated separately. In the displayed fit the smooth test space has rank $9$, consistent with nine smooth coefficients after the centring constraint, while penalization lowers its effective dimension.

The intercept estimate is $-0.13241$, with standard error $0.05426$, $t=-2.44$ and $p=0.0151$. Because the smooth is centred over the observations, the intercept is the overall fitted average, not necessarily the fitted response at $x=0$. Its test compares this average with zero. The significance-code legend maps progressively smaller $p$-values to stars; stars are an annotation, not an effect-size measure.

The smooth has $8.237$ effective degrees of freedom, displaying substantial nonlinearity: a centred purely linear term would use one. Est.rank=9 reports the rank used for its approximate significance calculation. The $p$-value below $2\times 10^{-16}$ is very strong evidence that the mean depends on $x$, against a zero smooth term. It does not mean the fitted curve is the exact generating function, or that lack of fit has been tested. The total effective dimension including the intercept is $1+8.237=9.237$.

The adjusted coefficient of determination $R^2$ of $0.983$ and deviance explained of $98.4\%$ indicate that nearly all variation about the response mean is explained. For this Gaussian model, deviance explained is $1-\mathrm{RSS}/\mathrm{SST}$. The scale estimate $0.96737$ estimates residual variance, corresponding to a residual standard deviation about $0.984$. The GCV score $0.99018$ is the prediction-oriented criterion used for smoothing selection, not a probability. It is consistent with the reported scale and effective dimension: $0.96737\times 401/(401-9.237)\simeq 0.99018$.

In the PDF's plot, the solid curve is the estimated centred smooth $f(x)$, and the dashed curves show pointwise uncertainty around that smooth, rather than prediction intervals for new noisy responses. The label s(x,8.24) gives its rounded effective degrees of freedom. The rug marks the observed covariate locations. Reading the endpoints givesThe smooth is increasing with an almost flat middle and one inflection, resembling a cubic. A simpler model to investigate is ordinary Gaussian polynomial regressionIt uses four mean coefficients rather than about nine effective ones. The apparent near-odd shape also makes ${\beta }_0+{\beta }_3{x}^3$ a plausible further reduction, but the linear and quadratic terms should be checked rather than discarded solely from the picture. The figure suggests this alternative; it does not supply coefficients or establish the true simulation model.

Condition on the training covariates, and let $m_i=\mathbb{E}{Y}_i$. The fresh response $Y_i^N$ has the same marginal distribution as $Y_i$ and is independent of the complete training response vector, hence independent of ${\hat{Y}}_i=\hat{f}(x_i)$. Assume the relevant second moments are finite. Expanding the squared errors givesThe first expectation averages over both training and new responses; the second averages over training responses only. Subtracting cancels both second-moment terms, leavingAveraging over the design points proves the covariance formula for prediction optimism:No linear fitting rule or Gaussian noise assumption is needed. The difference arises because a training response and its fitted value are dependent, whereas a fresh response and the fitted value are independent. Ordinary fitting often makes those covariances positive and the training error too favorable, explaining the name prediction optimism.

For example, if $\hat{Y}=HY$ is a fixed linear smoother and $\mathrm{Cov}(Y)={\sigma }^{2}I$, then $\mathrm{Cov}(Y_i,{\hat{Y}}_i)={\sigma }^2{H}_{ii}$. ThusThis links the optimism correction to effective degrees of freedom. For a data-selected fitting operator, its realized trace alone does not generally give the covariance penalty, since selecting the model uses the responses too.

An orthonormal multiresolution analysis is a sequence of closed subspaces $V_j\subset L^2(\mathbb{R})$ satisfying $V_j\subset V_{j+1}$, $\overline{{\bigcup }_j{V}_j}=L^2(\mathbb{R})$, ${\bigcap }_j{V}_j=\{0\}$, and $f\in V_j$ iff $f(2\cdot )\in V_{j+1}$. A scaling function $\varphi$ has integer translates forming an orthonormal basis of $V_0$, and its scaled translates ${\varphi }_{j,k}(x)=2^{j/2}\varphi (2^{j}x-k)$ span $V_j$. Thus $V_j$ contains information at resolution about $2^{-j}$.

The detail space $W_j=V_{j+1}\cap V_j^{\perp }$ is generated by the corresponding wavelet functions ${\psi }_{j,k}(x)=2^{j/2}\psi (2^{j}x-k)$. The orthogonal splitting $V_{j+1}=V_j\oplus W_j$ iterates toAccordingly a square-integrable signal has a coarse-and-detail wavelet series,and its wavelet projection onto $V_J$ keeps the coarse coefficients and details below level $J$. Increasing $J$ adds finer-scale oscillations and localized features. On the real line there are infinitely many translates at each level; for a bounded observation interval, with an appropriate interval or periodic construction, the relevant approximation dimension satisfies $d_J\asymp 2^J$. For the Haar wavelet on a unit interval this is exactly $2^J$, corresponding to a separate mean on each dyadic subinterval.

This hierarchy gives a concrete meaning to wavelet approximation complexity. Coarse resolution uses few coefficients and can miss sharp features; fine resolution uses more coefficients and can fit noise. By orthogonality, projection bias is the energy of the omitted details,For an orthonormal sequence model with independent coefficient noise of variance ${\sigma }^2/n$, estimating the $d_J$ retained coefficients without shrinkage givesThe first term decreases with resolution while the second increases, making the bias-variance tradeoff explicit. Equivalently, a fixed orthogonal regression projection of rank $d_J$ has effective degrees of freedom $d_J$ and prediction optimism $2{\sigma }^2{d}_J/n$ under independent equal-variance errors.

Wavelets also permit local rather than only global complexity control. Thresholding small estimated detail coefficients, for example by soft thresholding, retains strong local structure while suppressing fine-scale noise elsewhere. A sparse retained coefficient set can represent localized features much more economically than keeping every coefficient through a high level. Its locations and scales matter as well as its count. If that set is selected using the same noisy responses, however, selection adds adaptivity: its raw coefficient count is not automatically its covariance-based degrees of freedom. Resolution or threshold tuning should therefore account for prediction error, using an appropriate risk estimate or cross-validation, rather than only decreasing training error.

The trace of the smoother's influence matrix is its total effective degrees of freedom. Include the unpenalized intercept as well as both centered smooth terms:The Ref.df entries are for approximate significance calibration and must not be summed to obtain this trace. A consistency check using the generalized cross-validation form $nD/(n-\mathrm{tr}A{)}^2$ and scale estimate $D/(n-\mathrm{tr}A)$ gives $\mathrm{tr}A\approx 200(1-3.3506/3.4217)\approx 4.16$; the small difference is explained by rounding in the displayed quantities. The required trace is about $4.15$.

The original PDF's age smooth is nearly flat, its uncertainty band includes zero across the age range, and its approximate p-value is $0.859$. The exposure smooth is increasing and convex: it is fairly flat at low exposure and rises more steeply at high exposure. Its effective degrees of freedom $1.895$ suggest a modest departure from a straight line, rather than a highly oscillatory relationship.

A suitable simpler model is therefore the grouped quasibinomial regressionwith age omitted and dispersion estimated. A positive quadratic coefficient captures the convex exposure relationship; retaining the linear term respects the usual polynomial hierarchy. This model should be refitted, rather than assigning coefficients from the smooth plot. Compare its residual pattern and dispersion-adjusted fit with the additive model to check that the quadratic approximation is adequate. An exposure-only quadratic logit model is a justified parsimonious candidate, while the graph does not justify a sharp threshold or a zero-dispersion model.

The proposed generalized additive model retains the binary response and logit link, but replaces the linear age effect by a smooth function:A penalized cubic regression spline represents $f$; frailty remains a parametric coefficient. The plotted age smooth is centered, so its vertical values represent an age contribution to log odds, not probabilities or the entire fitted predictor.

The plot gives no evidence that the nonlinear model improves the mean structure. The fitted curve is essentially a straight descending line, and the label s(age,1) reports effective degrees of freedom close to one, consistent with a linear effect. The widening bands near the ends reflect sparse data and increasing uncertainty, not detected curvature. A linear-age logistic regression is therefore adequate on this evidence and simpler to interpret. A formal comparative claim would need a suitable test or predictive/model-selection criterion, but the figure supplies no reason to retain extra nonlinear flexibility.

Assume initially that the design points are distinct and ordered on $[a,b]$. The cubic smoothing spline minimizesover functions with an absolutely continuous first derivative and square-integrable second derivative. Dividing the residual term by $n$ simply changes the scaling assigned to $\lambda$. The minimum roughness property of the natural cubic spline interpolant reduces the problem to natural cubic splines with knots at the design points: among functions with any prescribed fitted values, that interpolant has the smallest integrated squared curvature. The minimizer is therefore a natural cubic smoothing spline, with natural second-derivative boundary conditions and linear tails when extended.

Choose a full basis $B_1,\dots ,B_n$ for this natural-spline space. Put $B_{ij}=B_j(X_i)$ and ${\Omega }_{jk}={\int }_a^b{B}_j^{\prime \prime }(t)B_k^{\prime \prime }(t)dt$. Writing $g(t)={\sum }_j{c}_j{B}_j(t)$ changes the objective toDifferentiation gives $(B^{T}B+\lambda \Omega )\hat{c}=B^{T}Y$. Although the curvature penalty has an affine nullspace, the full basis at distinct design points makes the combined matrix positive definite. HenceEquivalently, use the cardinal basis that interpolates the observations, making $B=I$ and $S_{\lambda }=(I+\lambda \Omega {)}^{-1}$. These are smoothing matrices, symmetric but generally not idempotent. Their effective degrees of freedom are $\mathrm{tr}(S_{\lambda })$. As $\lambda \downarrow 0$ the fit interpolates; as $\lambda \uparrow \infty$ it approaches ordinary least squares on the affine functions. With replicated predictors, use a design matrix for distinct knots and observation weights; the basis form remains valid when identifiable.

Useful choices include Leave-one-out cross-validation, generalized cross-validation, and estimation of smoothness by restricted maximum likelihood through a mixed-model representation. For fixed $\lambda$, the leave-one-out residual identity for a linear smoother givesMinimize these over a suitable range. Choose smoothing by predictive performance or a variance-component likelihood, not by arbitrarily forcing a visually appealing curve.

The null model's 99 residual degrees of freedom imply $n=100$. Each predictor has a one-degree-of-freedom linear contribution in the parametric table, plus the reported nonlinear contribution. Consequently the termwise effective degrees of freedom areInclude the intercept once, giving about $12.2$ total effective degrees of freedom. The more precise residual degrees of freedom in the output giveThere is no conflict: $8.8$ and $0.4$ are rounded, so they cannot recover the precise total. Individual unrounded termwise values are not provided.

The package's residual mean-square estimate of error variance ismatching the rounded $10328$ in the table. The nonlinear experience contribution is highly significant, while the smaller nonlinear education contribution has approximate $p=0.03923$.

This calculation uses the effective residual degrees of freedom reported by this fit. For a general non-idempotent linear smoother $S$, the noise contribution to expected RSS is ${\sigma }^2\mathrm{tr}[(I-S{)}^T(I-S)]$, and the expectation can additionally contain squared smoothing bias. Thus dividing by $n-\mathrm{tr}(S)$ is not a universal exact-unbiasedness theorem. The output's separation of linear and nonlinear term degrees of freedom is documented at cran.r-project.org/web/packages/gam/gam.pdf.

The generalized additive mixed model fitswhere $u_j$ denotes the furnace random intercept and $f$ is a centered penalized cubic regression spline. Its wiggly components have a mixed-model representation; there is no explicit furnace-specific random slope in this formula.

The plotted smooth gives no evidence that a nonparametric fixed effect is needed. It is essentially a straight rising line, and s(stir,1) indicates effective degrees of freedom close to one. The wider bands at extreme stirring rates reflect uncertainty, not detected nonlinearity. A linear fixed stirring effect is adequate on this evidence.

If the three supplied Akaike information criterion values are on a common valid likelihood basis, choose strength_model3, the fixed linear slope plus random-intercept model: its AIC $206.6408$ is smallest, versus $208.3243$ for the additional random slope and $210.3425$ for the smooth. The differences are $1.6835$ and $3.7017$, respectively, so especially the random-slope comparison represents modest support rather than decisive evidence. Retaining the population slope is consistent with part (c); this choice removes only its random variation.

There is a likelihood-basis caveat in the literal printed commands. lmer fits by REML by default, whereas the Gaussian gamm call defaults to ML; direct AIC on these objects need not compare the same type of likelihood. Moreover the earlier ML ANOVA prints $210.22$ for strength_model, whereas this table prints $208.3243$, so these numbers should not silently be treated as one identical ML fit. To obtain a defensible common comparison, fit all candidates to the same observations by ML:The supplied ranking favours the simpler random-intercept model; the displayed original calls alone do not certify that ranking as a consistent ML comparison. This distinction is particularly relevant when the smooth changes the fixed-effect space. The Gaussian gamm ML default and mixed-model representation are documented at stat.ethz.ch/R-manual/R-devel/library/mgcv/html/gamm.html.

For $\lambda >0$, minimize the smoothing spline objective over the Sobolev space of functions with square-integrable second derivative on the observed interval:The roughness penalty measures curvature; its null space consists of affine functions. To see why the minimizer is a natural cubic spline, let $g$ be the natural spline interpolating the values of any competitor $m$ at its distinct observation locations, and put $h=m-g$. Integration by parts twice on each knot interval gives $\int g^{\prime \prime }{h}^{\prime \prime }=0$: $g^{\prime \prime \prime \prime }=0$ on each interval, $h$ vanishes at the knots, $g^{\prime \prime }$ is continuous, and $g^{\prime \prime }$ vanishes at the outer endpoints. Hencewhile both functions have the same residual sum of squares. This proves the reduction to a finite-dimensional spline space.

For distinct $X_i$ and $n\ge 2$, use its $n$-dimensional basis, write $B_{ij}=b_j(X_i)$, and define the positive semidefinite matrixThe penalized least squares problem is $\Vert Y-B\beta {\Vert }^2+\lambda {\beta }^T\Omega \beta$, soUse linear continuation outside the observation interval. The fitted-value matrix is $S_{\lambda }=B(B^{T}B+\lambda \Omega {)}^{-1}{B}^T$; its trace gives the effective degrees of freedom. As $\lambda$ decreases to zero, the natural spline approaches interpolation, with trace $n$; as it grows without bound, the fit approaches ordinary least squares on $1,x$, with trace two. For $\lambda =0$ the criterion alone does not determine values between observations; the natural interpolant is the limiting convention. Repeated observation locations are combined by using distinct knots and their multiplicities as weights, rather than pretending the spline design has $n$ independent interpolation values.

Because no family is specified, the command fits a Gaussian identity-link generalized additive model:The $s_j$ are centered penalized regression splines with cubic regression bases; centering separates them from the intercept. The age curve in the original figure falls to a minimum near age 25, then rises markedly, particularly from about 40 onwards. Its effective degrees of freedom are $4.05$, reflecting curvature. The white-cell curve has effective degrees of freedom one and is nearly flat with a slight positive slope; the pointwise uncertainty bands are consistent with no substantial effect. These plots display centered partial mean contributions, not numbers of prescriptions by themselves.

A simpler mean model keeps the age regression spline and replaces the white-cell smooth by a linear term. A separate issue is that the response is a nonnegative count: a Poisson regression with logarithmic link function respects that support and positivity of the mean. A reasonable candidate isIf diagnostics show overdispersion, one can fitusing a negative binomial regression. Alternatively, if a Gaussian approximation is adequate, the minimal simplification is gam(npres ~ s(age, bs="cr") + wbc). These are candidates to compare by diagnostics and validation, rather than guaranteed improvements from the partial plots alone. Retain the curved age effect; consider a linear or removable white-cell effect, and assess an appropriate count family. Refit before interpreting the old curves on a new link scale.

The command fits a Gaussian generalized additive mixed modelThe population growth curve $s$ is a centered penalized regression spline with a cubic regression basis. The incubator term remains a random slope without a random intercept, because ~0+hours removes the intercept. A population linear component is allowed within the smooth's unpenalized part; curvature is penalized rather than being required.

A gamm fit returns a list with gam and lme components. Plot the population smooth and pointwise uncertainty bands withInspect whether the fitted curve departs materially from an affine function, particularly in regions supported by observations. The effective degrees of freedom near one suggest that the penalization has selected an approximately linear centered smooth. Bands and partial residuals help distinguish convincing curvature from noisy deviations, although pointwise bands are not a simultaneous test of linearity. A formal comparison would need to account for both smoothing selection and the incubator dependence. The smooth plot tests the visual plausibility of a common straight growth curve while retaining incubator-specific slope variation.

Put $f=(f(x_1),\dots ,f(x_n){)}^T$, so $Y=f+\epsilon$ and $Y^{\ast }=f+{\epsilon }^{\ast }$ with independent noise vectors having covariance matrix $I_n$. For any deterministic linear smoother $H$,whereas independence givesExpanding the first trace shows that the second expression exceeds the first by $2\mathrm{tr}(H)$, proving the identity.

For ridge regression,Its effective degrees of freedom are $\mathrm{tr}(H_{\lambda })$. Thus training error is optimistically biased for independent-copy prediction error by $2\mathrm{tr}(H_{\lambda })/n$. The graph exhibits exactly this effect: the dashed training curve keeps falling as $\lambda$ decreases, while the solid test curve eventually rises through overfitting.

For $\lambda \ge 0$, a full-column-rank design matrix $B$, and a positive semidefinite matrix $\Omega$, minimize $\Vert y-B\beta {\Vert }^2+\lambda {\beta }^T\Omega \beta$. Differentiation gives $(B^{T}B+\lambda \Omega )\hat{\beta }=B^{T}y$. The fitted-value matrix is $B(B^{T}B+\lambda \Omega {)}^{-1}{B}^T$, whose trace measures the effective degrees of freedom.