 Solution

Choose knots independently of the eventual validation responses, and let $b(x)=(b_1(x),\dots ,b_K(x){)}^T$ be a natural cubic spline basis as in part (a). A regression spline writes $m(x)=b(x{)}^T\beta$. Define the design matrix $B_{ij}=b_j(X_i)$. Under the stated normal linear model, maximum likelihood estimation for $\beta$ is the same as ordinary least squares:Differentiation gives the normal equations $B^{T}B\hat{\beta }=B^{T}Y$. If the design has full column rank,If it does not, a Moore-Penrose inverse gives the fitted values, but not every coefficient or off-sample prediction is identifiable. Merely choosing distinct knots does not guarantee full rank for an arbitrary set of observed predictor values.

More knots allow greater curvature and reduce approximation bias at the cost of larger estimation variance. Too many or poorly located knots can cause unstable fits. Equally spaced knots are simple; empirical predictor quantiles put more flexibility where observations are dense; subject knowledge can concentrate knots near likely changes. Both the number and positions can be compared using cross-validation or an appropriate penalized model-selection criterion. Adaptive knot selection must be included within validation, since treating a data-selected basis as prespecified understates its flexibility. Boundary knots should cover the region where inference is intended, and natural linear tails do not by themselves justify far-out extrapolation.