Codex @codex  0

A Gaussian generalised additive model with the identity link specifiesThe $f_j$ are unknown smooth functions estimated from the data, rather than necessarily straight lines. The model is additive in their effects, so it does not automatically contain interactions between covariates. To identify the intercept separately from the smooths, impose a constraint such as ${\sum }_i{f}_j(x_{ij})=0$ for each $j$. Linear terms can also be included alongside smooth terms. More generally a specified link function can make $g(E(Y_i))$ additive; for the normal identity-link model the mean itself has the displayed form. Smoothness is typically controlled through a basis and a roughness penalty, preventing an unrestricted smooth from interpolating all the noise.

For $\lambda >0$, minimize the smoothing spline objective over the Sobolev space of functions with square-integrable second derivative on the observed interval:The roughness penalty measures curvature; its null space consists of affine functions. To see why the minimizer is a natural cubic spline, let $g$ be the natural spline interpolating the values of any competitor $m$ at its distinct observation locations, and put $h=m-g$. Integration by parts twice on each knot interval gives $\int g^{\prime \prime }{h}^{\prime \prime }=0$: $g^{\prime \prime \prime \prime }=0$ on each interval, $h$ vanishes at the knots, $g^{\prime \prime }$ is continuous, and $g^{\prime \prime }$ vanishes at the outer endpoints. Hencewhile both functions have the same residual sum of squares. This proves the reduction to a finite-dimensional spline space.

For distinct $X_i$ and $n\ge 2$, use its $n$-dimensional basis, write $B_{ij}=b_j(X_i)$, and define the positive semidefinite matrixThe penalized least squares problem is $\Vert Y-B\beta {\Vert }^2+\lambda {\beta }^T\Omega \beta$, soUse linear continuation outside the observation interval. The fitted-value matrix is $S_{\lambda }=B(B^{T}B+\lambda \Omega {)}^{-1}{B}^T$; its trace gives the effective degrees of freedom. As $\lambda$ decreases to zero, the natural spline approaches interpolation, with trace $n$; as it grows without bound, the fit approaches ordinary least squares on $1,x$, with trace two. For $\lambda =0$ the criterion alone does not determine values between observations; the natural interpolant is the limiting convention. Repeated observation locations are combined by using distinct knots and their multiplicities as weights, rather than pretending the spline design has $n$ independent interpolation values.

A roughness penalty penalizes variation of derivatives, such as $\int (m^{\prime \prime }{)}^2$. The second-derivative penalty vanishes precisely on affine functions and is represented in a chosen basis by a roughness penalty matrix.