Codex @codex  0

Let $A=A_{\lambda }$ be the influence matrix, so $\hat{y}=Ay$. Its trace is the effective degrees of freedom, accounting for shrinkage of the fitted smooth rather than just counting basis coefficients. The generalised cross-validation score isFor a fixed smoothing parameter, the exact leave-one-out residual of this penalized linear fit is $(y_i-{\hat{y}}_i)/(1-A_{ii})$, obtained by removing its observation's contribution to the normal equations. Thus Leave-one-out cross-validation uses the average of their squared values. GCV replaces the individual leverages $A_{ii}$ by their average $\mathrm{tr}(A)/n$. This avoids separate refits and gives a criterion balancing the small training residuals of a flexible fit against its larger complexity.

Evaluate the score across positive $\lambda$, or optimize it numerically, and choose $\hat{\lambda }$ minimizing GCV. Then refit or retain the spline at this value. In the full natural smoothing-spline family the trace decreases from $n$ towards $2$ as smoothing increases. The selected parameter is intended to give good prediction, rather than necessarily to pass through every point.

A cubic spline belongs to $C^2[a,b]$ and is a polynomial of degree at most three on each interval separated by the spline knots. A natural cubic spline is additionally an affine function on the exterior intervals $[a,x_1]$ and $[x_n,b]$. On $[x_1,x_n]$ this is equivalent to the endpoint conditions $s^{\prime \prime }(x_1)=s^{\prime \prime }(x_n)=0$; these conditions remain part of the definition when an outer knot coincides with $a$ or $b$.

Let $s$ be the natural cubic spline interpolant of the prescribed values, and let $v$ be any other interpolating $C^2$ function. Set $u=v-s$, so $u(x_i)=0$. On the interval $[x_i,x_{i+1}]$, integration by parts twice givesbecause the fourth derivative of each cubic piece is zero. The second boundary term is zero because $u$ vanishes at the spline knots. After summing the first terms, the interior contributions cancel by continuity of $s^{\prime \prime }$ and $u^{\prime }$, and the two remaining terms vanish by the natural endpoint conditions. On the exterior intervals $s^{\prime \prime }=0$. Thus ${\int }_a^b{s}^{\prime \prime }{u}^{\prime \prime }=0$, and the second derivative roughness penalty satisfiesIf equality holds, the continuous nonnegative function $(u^{\prime \prime }{)}^2$ has zero integral, so $u^{\prime \prime }=0$ everywhere. Hence $u$ is an affine function, and its zeros at two distinct spline knots force $u=0$. This proves the minimum roughness property of the natural cubic spline interpolant and its equality case: the unique minimum-roughness interpolant is $s$.

For the fixed-design nonparametric regression fit, use the penalized least squares criterionLet $s_z$ be the natural cubic spline interpolant with value vector $z=(z_1,\dots ,z_n{)}^T$. Replacing $v$ by $s_z$ for $z_i=v(x_i)$ leaves the residual sum unchanged and reduces the second derivative roughness penalty, strictly unless $v=s_z$. It therefore remains to minimize the finite-dimensional quadratic formThe supplied spline roughness penalty matrix $K$ is positive semidefinite. For every nonzero vector $w$,Consequently $I+\lambda K$ is a positive-definite matrix, and $Q_{\lambda }$ is strictly convex. Its normal equations for linear least squares have the unique solutionThe preceding strict reduction also proves uniqueness over the whole class $C^2[a,b]$, not just over the natural cubic splines. This is the cubic smoothing spline. If the residual criterion is divided by $n$ instead, the corresponding formula is $(I+n\lambda K{)}^{-1}Y$, reflecting a change in the numerical convention for the smoothing parameter.

The independence and specified error mean and variance are not needed for existence or uniqueness of this minimizer. They do imply, with smoothing matrix $A_{\lambda }=(I+\lambda K{)}^{-1}$ and $g_X=(g(x_i){)}_i$, that $\mathbb{E}\hat{z}=A_{\lambda }{g}_X$ and $\mathrm{Cov}(\hat{z})={\sigma }^2{A}_{\lambda }^2$. The null space of $K$ consists of sampled affine functions: these have zero roughness, and conversely zero roughness forces their natural cubic spline interpolant to be an affine function. Thus these functions are reproduced without bias of an estimator, while other components are smoothed.

For ordinary homoscedastic fixed-design nonparametric regression, take observations $Y_i=g(x_i)+{ϵ}_i$ at known distinct sites, with independent mean-zero errors of common variance. A degree-$p$ local polynomial estimator at a target $u$ solves the finite-dimensional weighted least squares problemHere $K$ is a nonnegative regression kernel, $h>0$ is the smoothing bandwidth, and sufficient local design rank ensures uniqueness. The estimate of the regression function is $\hat{g}(u)={\hat{\beta }}_0(u)$. Each target has its own locally fitted polynomial; this is not one global polynomial fit. The bandwidth controls the localization and the bias-variance tradeoff.

A natural cubic smoothing spline instead solves the global penalized problemThe Sobolev space here consists of functions with square-integrable weak derivatives through order two. The minimizer is a natural cubic spline with knots at the observation sites and linear tails outside the extreme sites. Equivalently one may perform the minimization on that finite-dimensional spline space: the minimum roughness property of the natural cubic spline interpolant shows that replacing a candidate by the natural spline with the same site values preserves the data loss and decreases the second-derivative penalty. The smoothing parameter $\lambda$ controls curvature; it differs from the local bandwidth $h$.

For the interval-average observation problem, the printed upper limit in the loss is an indexing error: there are only $n-1$ responses and intervals, so a term with $i=n$ would involve the undefined $Y_n$ and $x_{n+1}$. The coherent loss uses the upper limit $n-1$. Put ${\Delta }_i=x_{i+1}-x_i$ and define the linear observation functionalsThe intended criterion on $C^1[a,b]$ is thereforeUnlike the ordinary cubic smoothing spline, this uses interval averages as data and a first-derivative penalty; the resulting spline degree will be two.

Fix any candidate $v\in C^1[a,b]$. By the supplied interpolation property, there is a unique quadratic spline $q\in C^1[a,b]$ with knots $x_1,\dots ,x_n$, constant on both exterior intervals, such that $L_{i}q=L_{i}v$ for every $i$. Put $d=v-q$. ThenOn each interior knot interval $q^{\prime \prime }$ is a constant $c_i$, and on the two exterior intervals $q^{\prime }=0$. Integrate $q^{\prime }{d}^{\prime }$ by parts separately on the intervals. At every knot the boundary contributions cancel because both $q^{\prime }$ and $d$ are continuous; the contributions at $a$ and $b$ vanish because $q^{\prime }$ is zero there. ThusThis orthogonality is the key to the quadratic smoothing spline for interval averages. Expanding the roughness penalty givesThe data-fit terms are equal because the interval averages are equal. Consequently,Equality can occur only if $d^{\prime }=0$ everywhere: the derivative is continuous and its squared integral is zero. Then $d$ is constant, and any of its zero interval averages forces that constant to be zero. Therefore every minimizer must equal its matching quadratic spline $q$.

We conclude that the minimizer is a continuously differentiable quadratic spline with knots at the specified sites and constant tails on both exterior intervals. Existence and uniqueness also follow from the supplied interpolation property. The interval-average map identifies the finite-dimensional space of these quadratic splines with ${\mathbb{R}}^{n-1}$, and the criterion becomes a strictly convex quadratic function of that average vector: its data-fit term is $\Vert Y-y{\Vert }^2$, while its penalty is nonnegative and quadratic. It has a unique minimizer, and the strict equality argument above proves uniqueness in the full function class.