= Solution
Choose knots independently of the eventual validation responses, and let $b(x)=(b_1(x),\ldots,b_K(x))^T$ be a <natural cubic spline> <basis> as in part (a). A <regression spline> writes $m(x)=b(x)^T\beta$. Define the <design matrix> $B_{ij}=b_j(X_i)$. Under the stated <normal linear model>, <maximum likelihood estimation> for $\beta$ is the same as <ordinary least squares>:
$$
\widehat\beta=\arg\min_\beta\|Y-B\beta\|^2.
$$
Differentiation gives the <normal equations> $B^TB\widehat\beta=B^TY$. If the design has full column rank,
$$
\boxed{\widehat m(x)=b(x)^T(B^TB)^{-1}B^TY.}
$$
If it does not, a <Moore-Penrose inverse> gives the fitted values, but not every coefficient or off-sample prediction is identifiable. Merely choosing distinct knots does not guarantee full rank for an arbitrary set of observed predictor values.
More knots allow greater curvature and reduce approximation <bias> at the cost of larger estimation <variance>. Too many or poorly located knots can cause unstable fits. Equally spaced knots are simple; empirical predictor <quantiles> put more flexibility where observations are dense; subject knowledge can concentrate knots near likely changes. Both the number and positions can be compared using <cross-validation> or an appropriate penalized model-selection criterion. Adaptive knot selection must be included within validation, since treating a data-selected basis as prespecified understates its flexibility. Boundary knots should cover the region where inference is intended, and natural linear tails do not by themselves justify far-out extrapolation.
Back to article page
