= Solution
A <cubic spline> belongs to $C^2[a,b]$ and is a <polynomial> of degree at most three on each interval separated by the <spline knots>. A <natural cubic spline> is additionally an <affine function> on the exterior intervals $[a,x_1]$ and $[x_n,b]$. On $[x_1,x_n]$ this is equivalent to the endpoint conditions $s''(x_1)=s''(x_n)=0$; these conditions remain part of the definition when an outer knot coincides with $a$ or $b$.
Let $s$ be the <natural cubic spline interpolant> of the prescribed values, and let $v$ be any other interpolating $C^2$ function. Set $u=v-s$, so $u(x_i)=0$. On the interval $[x_i,x_{i+1}]$, <integration by parts> twice gives
$$
\int_{x_i}^{x_{i+1}}s''u''=[s''u']_{x_i}^{x_{i+1}}-[s'''u]_{x_i}^{x_{i+1}},
$$
because the fourth <derivative> of each cubic piece is zero. The second boundary term is zero because $u$ vanishes at the <spline knots>. After summing the first terms, the interior contributions cancel by <continuity> of $s''$ and $u'$, and the two remaining terms vanish by the natural endpoint conditions. On the exterior intervals $s''=0$. Thus $\int_a^b s''u''=0$, and the <second derivative roughness penalty> satisfies
$$
R(v'')=R(s'')+R(u'')\geq R(s'').
$$
If equality holds, the continuous nonnegative function $(u'')^2$ has zero integral, so $u''=0$ everywhere. Hence $u$ is an <affine function>, and its zeros at two distinct <spline knots> force $u=0$. This proves the <minimum roughness property of the natural cubic spline interpolant> and its equality case: \b[the unique minimum-roughness interpolant is $s$].
For the <fixed-design nonparametric regression> fit, use the <penalized least squares> criterion
$$
\boxed{S_\lambda(v)=\sum_{i=1}^n\{Y_i-v(x_i)\}^2+\lambda\int_a^b v''(x)^2\,dx.}
$$
Let $s_z$ be the <natural cubic spline interpolant> with value <vector> $z=(z_1,\ldots,z_n)^T$. Replacing $v$ by $s_z$ for $z_i=v(x_i)$ leaves the residual sum unchanged and reduces the <second derivative roughness penalty>, strictly unless $v=s_z$. It therefore remains to minimize the finite-dimensional <quadratic form>
$$
Q_\lambda(z)=\|Y-z\|^2+\lambda z^TKz.
$$
The supplied <spline roughness penalty matrix> $K$ is <positive semidefinite>. For every nonzero <vector> $w$,
$$
w^T(I+\lambda K)w=\|w\|^2+\lambda w^TKw>0.
$$
Consequently $I+\lambda K$ is a <positive-definite matrix>, and $Q_\lambda$ is <strictly convex>. Its <normal equations for linear least squares> have the unique solution
$$
\boxed{\widehat z=(I+\lambda K)^{-1}Y,\qquad \widehat g_\lambda=s_{\widehat z}.}
$$
The preceding strict reduction also proves uniqueness over the whole class $C^2[a,b]$, not just over the <natural cubic splines>. This is the <cubic smoothing spline>. If the residual criterion is divided by $n$ instead, the corresponding formula is $(I+n\lambda K)^{-1}Y$, reflecting a change in the numerical convention for the <smoothing parameter>.
The <independence> and specified error <mean> and <variance> are not needed for existence or uniqueness of this minimizer. They do imply, with <smoothing matrix> $A_\lambda=(I+\lambda K)^{-1}$ and $g_X=(g(x_i))_i$, that $\mathbb E\widehat z=A_\lambda g_X$ and $\operatorname{Cov}(\widehat z)=\sigma^2A_\lambda^2$. The <null space> of $K$ consists of sampled <affine functions>: these have zero roughness, and conversely zero roughness forces their <natural cubic spline interpolant> to be an <affine function>. Thus these functions are reproduced without <bias of an estimator>, while other components are smoothed.
Back to article page
