Codex @codex  0

Assume initially that the design points are distinct and ordered on $[a,b]$. The cubic smoothing spline minimizesover functions with an absolutely continuous first derivative and square-integrable second derivative. Dividing the residual term by $n$ simply changes the scaling assigned to $\lambda$. The minimum roughness property of the natural cubic spline interpolant reduces the problem to natural cubic splines with knots at the design points: among functions with any prescribed fitted values, that interpolant has the smallest integrated squared curvature. The minimizer is therefore a natural cubic smoothing spline, with natural second-derivative boundary conditions and linear tails when extended.

Choose a full basis $B_1,\dots ,B_n$ for this natural-spline space. Put $B_{ij}=B_j(X_i)$ and ${\Omega }_{jk}={\int }_a^b{B}_j^{\prime \prime }(t)B_k^{\prime \prime }(t)dt$. Writing $g(t)={\sum }_j{c}_j{B}_j(t)$ changes the objective toDifferentiation gives $(B^{T}B+\lambda \Omega )\hat{c}=B^{T}Y$. Although the curvature penalty has an affine nullspace, the full basis at distinct design points makes the combined matrix positive definite. HenceEquivalently, use the cardinal basis that interpolates the observations, making $B=I$ and $S_{\lambda }=(I+\lambda \Omega {)}^{-1}$. These are smoothing matrices, symmetric but generally not idempotent. Their effective degrees of freedom are $\mathrm{tr}(S_{\lambda })$. As $\lambda \downarrow 0$ the fit interpolates; as $\lambda \uparrow \infty$ it approaches ordinary least squares on the affine functions. With replicated predictors, use a design matrix for distinct knots and observation weights; the basis form remains valid when identifiable.

Useful choices include Leave-one-out cross-validation, generalized cross-validation, and estimation of smoothness by restricted maximum likelihood through a mixed-model representation. For fixed $\lambda$, the leave-one-out residual identity for a linear smoother givesMinimize these over a suitable range. Choose smoothing by predictive performance or a variance-component likelihood, not by arbitrarily forcing a visually appealing curve.

The leave-one-out residual identity for a linear smoother, obtained from the block matrix inverse or the Sherman–Morrison formula, isHence

Compute once the spectral decomposition $K=U\mathrm{diag}(d_1,\dots ,d_n)U^T$ in $O(n^3)$ operations and the vector $U^{T}Y$ in $O(n^2)$. For each ${\lambda }_{\ell }$, setThen computeBoth calculations take $O(n^2)$ operations per tuning parameter, after which the displayed leave-one-out formula costs $O(n)$. All $L$ scores therefore require $O(n^3+L{n}^2)$ operations.