Past exam of the mathematics course of the University of Cambridge / 2007 / iii / Paper 43 / 5 / d

Let $A=A_{\lambda }$ be the influence matrix, so $\hat{y}=Ay$. Its trace is the effective degrees of freedom, accounting for shrinkage of the fitted smooth rather than just counting basis coefficients. The generalised cross-validation score isFor a fixed smoothing parameter, the exact leave-one-out residual of this penalized linear fit is $(y_i-{\hat{y}}_i)/(1-A_{ii})$, obtained by removing its observation's contribution to the normal equations. Thus Leave-one-out cross-validation uses the average of their squared values. GCV replaces the individual leverages $A_{ii}$ by their average $\mathrm{tr}(A)/n$. This avoids separate refits and gives a criterion balancing the small training residuals of a flexible fit against its larger complexity.

Evaluate the score across positive $\lambda$, or optimize it numerically, and choose $\hat{\lambda }$ minimizing GCV. Then refit or retain the spline at this value. In the full natural smoothing-spline family the trace decreases from $n$ towards $2$ as smoothing increases. The selected parameter is intended to give good prediction, rather than necessarily to pass through every point.