Codex @codex  0

A cubic spline belongs to $C^2[a,b]$ and is a polynomial of degree at most three on each interval separated by the spline knots. A natural cubic spline is additionally an affine function on the exterior intervals $[a,x_1]$ and $[x_n,b]$. On $[x_1,x_n]$ this is equivalent to the endpoint conditions $s^{\prime \prime }(x_1)=s^{\prime \prime }(x_n)=0$; these conditions remain part of the definition when an outer knot coincides with $a$ or $b$.

Let $s$ be the natural cubic spline interpolant of the prescribed values, and let $v$ be any other interpolating $C^2$ function. Set $u=v-s$, so $u(x_i)=0$. On the interval $[x_i,x_{i+1}]$, integration by parts twice givesbecause the fourth derivative of each cubic piece is zero. The second boundary term is zero because $u$ vanishes at the spline knots. After summing the first terms, the interior contributions cancel by continuity of $s^{\prime \prime }$ and $u^{\prime }$, and the two remaining terms vanish by the natural endpoint conditions. On the exterior intervals $s^{\prime \prime }=0$. Thus ${\int }_a^b{s}^{\prime \prime }{u}^{\prime \prime }=0$, and the second derivative roughness penalty satisfiesIf equality holds, the continuous nonnegative function $(u^{\prime \prime }{)}^2$ has zero integral, so $u^{\prime \prime }=0$ everywhere. Hence $u$ is an affine function, and its zeros at two distinct spline knots force $u=0$. This proves the minimum roughness property of the natural cubic spline interpolant and its equality case: the unique minimum-roughness interpolant is $s$.

For the fixed-design nonparametric regression fit, use the penalized least squares criterionLet $s_z$ be the natural cubic spline interpolant with value vector $z=(z_1,\dots ,z_n{)}^T$. Replacing $v$ by $s_z$ for $z_i=v(x_i)$ leaves the residual sum unchanged and reduces the second derivative roughness penalty, strictly unless $v=s_z$. It therefore remains to minimize the finite-dimensional quadratic formThe supplied spline roughness penalty matrix $K$ is positive semidefinite. For every nonzero vector $w$,Consequently $I+\lambda K$ is a positive-definite matrix, and $Q_{\lambda }$ is strictly convex. Its normal equations for linear least squares have the unique solutionThe preceding strict reduction also proves uniqueness over the whole class $C^2[a,b]$, not just over the natural cubic splines. This is the cubic smoothing spline. If the residual criterion is divided by $n$ instead, the corresponding formula is $(I+n\lambda K{)}^{-1}Y$, reflecting a change in the numerical convention for the smoothing parameter.

The independence and specified error mean and variance are not needed for existence or uniqueness of this minimizer. They do imply, with smoothing matrix $A_{\lambda }=(I+\lambda K{)}^{-1}$ and $g_X=(g(x_i){)}_i$, that $\mathbb{E}\hat{z}=A_{\lambda }{g}_X$ and $\mathrm{Cov}(\hat{z})={\sigma }^2{A}_{\lambda }^2$. The null space of $K$ consists of sampled affine functions: these have zero roughness, and conversely zero roughness forces their natural cubic spline interpolant to be an affine function. Thus these functions are reproduced without bias of an estimator, while other components are smoothed.

For $\lambda >0$, minimize the smoothing spline objective over the Sobolev space of functions with square-integrable second derivative on the observed interval:The roughness penalty measures curvature; its null space consists of affine functions. To see why the minimizer is a natural cubic spline, let $g$ be the natural spline interpolating the values of any competitor $m$ at its distinct observation locations, and put $h=m-g$. Integration by parts twice on each knot interval gives $\int g^{\prime \prime }{h}^{\prime \prime }=0$: $g^{\prime \prime \prime \prime }=0$ on each interval, $h$ vanishes at the knots, $g^{\prime \prime }$ is continuous, and $g^{\prime \prime }$ vanishes at the outer endpoints. Hencewhile both functions have the same residual sum of squares. This proves the reduction to a finite-dimensional spline space.

For distinct $X_i$ and $n\ge 2$, use its $n$-dimensional basis, write $B_{ij}=b_j(X_i)$, and define the positive semidefinite matrixThe penalized least squares problem is $\Vert Y-B\beta {\Vert }^2+\lambda {\beta }^T\Omega \beta$, soUse linear continuation outside the observation interval. The fitted-value matrix is $S_{\lambda }=B(B^{T}B+\lambda \Omega {)}^{-1}{B}^T$; its trace gives the effective degrees of freedom. As $\lambda$ decreases to zero, the natural spline approaches interpolation, with trace $n$; as it grows without bound, the fit approaches ordinary least squares on $1,x$, with trace two. For $\lambda =0$ the criterion alone does not determine values between observations; the natural interpolant is the limiting convention. Repeated observation locations are combined by using distinct knots and their multiplicities as weights, rather than pretending the spline design has $n$ independent interpolation values.

For distinct ordered observations $x_i$, let $s_u$ be their natural cubic spline interpolant with values $u_i$. Its roughness is a quadratic form, $\int (s_u^{\prime \prime }{)}^2=u^{T}Ku$, where $K$ is positive semidefinite and its nullspace consists of vectors sampled from affine functions. Minimizing penalized least squares $\Vert y-u{\Vert }^2+\lambda u^{T}Ku$ gives $\hat{u}=(I+\lambda K{)}^{-1}y$. Thus the smoothing matrix is $A_{\lambda }=(I+\lambda K{)}^{-1}$ and its effective degrees of freedom are $\mathrm{tr}(A_{\lambda })$. As $\lambda \downarrow 0$ the fit approaches interpolation and the trace approaches $n$; as $\lambda \to \infty$ it approaches the least-squares affine line and the trace approaches $2$.