Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 206 / 3

With $K\ge 2$ distinct knots ${\kappa }_1<\cdots <{\kappa }_K$, a natural cubic spline is a $C^2$ function on the real line, cubic on each interval between consecutive knots and affine outside the two outer knots. The latter condition gives $m^{\prime \prime }({\kappa }_1)=m^{\prime \prime }({\kappa }_K)=0$.

There are $4(K-1)$ coefficients for the interior cubic pieces. Matching values and the first two derivatives at the $K-2$ interior knots imposes $3(K-2)$ independent constraints, and the two natural endpoint conditions impose two more. The affine continuations are then determined by endpoint values and slopes. The dimension of a vector space is thereforeOne explicit basis uses $1,x$ and $d_j(x)-d_{K-1}(x)$ for $j=1,\dots ,K-2$, whereThe cubic and quadratic terms cancel in each difference above ${\kappa }_K$, making it affine there; below ${\kappa }_1$ the truncated powers vanish. The changes in third derivative at individual knots show independence of these basis functions. This count uses the linear independence of the spline constraints and counts the listed outer knots as boundary knots. If a convention instead lists $K$ interior knots and two additional boundaries, the dimension is $K+2$. With only one listed knot and linearity on both sides, the $C^2$ condition forces a globally affine function, so the exceptional dimension is two.

Choose knots independently of the eventual validation responses, and let $b(x)=(b_1(x),\dots ,b_K(x){)}^T$ be a natural cubic spline basis as in part (a). A regression spline writes $m(x)=b(x{)}^T\beta$. Define the design matrix $B_{ij}=b_j(X_i)$. Under the stated normal linear model, maximum likelihood estimation for $\beta$ is the same as ordinary least squares:Differentiation gives the normal equations $B^{T}B\hat{\beta }=B^{T}Y$. If the design has full column rank,If it does not, a Moore-Penrose inverse gives the fitted values, but not every coefficient or off-sample prediction is identifiable. Merely choosing distinct knots does not guarantee full rank for an arbitrary set of observed predictor values.

More knots allow greater curvature and reduce approximation bias at the cost of larger estimation variance. Too many or poorly located knots can cause unstable fits. Equally spaced knots are simple; empirical predictor quantiles put more flexibility where observations are dense; subject knowledge can concentrate knots near likely changes. Both the number and positions can be compared using cross-validation or an appropriate penalized model-selection criterion. Adaptive knot selection must be included within validation, since treating a data-selected basis as prespecified understates its flexibility. Boundary knots should cover the region where inference is intended, and natural linear tails do not by themselves justify far-out extrapolation.

For $\lambda >0$, minimize the smoothing spline objective over the Sobolev space of functions with square-integrable second derivative on the observed interval:The roughness penalty measures curvature; its null space consists of affine functions. To see why the minimizer is a natural cubic spline, let $g$ be the natural spline interpolating the values of any competitor $m$ at its distinct observation locations, and put $h=m-g$. Integration by parts twice on each knot interval gives $\int g^{\prime \prime }{h}^{\prime \prime }=0$: $g^{\prime \prime \prime \prime }=0$ on each interval, $h$ vanishes at the knots, $g^{\prime \prime }$ is continuous, and $g^{\prime \prime }$ vanishes at the outer endpoints. Hencewhile both functions have the same residual sum of squares. This proves the reduction to a finite-dimensional spline space.

For distinct $X_i$ and $n\ge 2$, use its $n$-dimensional basis, write $B_{ij}=b_j(X_i)$, and define the positive semidefinite matrixThe penalized least squares problem is $\Vert Y-B\beta {\Vert }^2+\lambda {\beta }^T\Omega \beta$, soUse linear continuation outside the observation interval. The fitted-value matrix is $S_{\lambda }=B(B^{T}B+\lambda \Omega {)}^{-1}{B}^T$; its trace gives the effective degrees of freedom. As $\lambda$ decreases to zero, the natural spline approaches interpolation, with trace $n$; as it grows without bound, the fit approaches ordinary least squares on $1,x$, with trace two. For $\lambda =0$ the criterion alone does not determine values between observations; the natural interpolant is the limiting convention. Repeated observation locations are combined by using distinct knots and their multiplicities as weights, rather than pretending the spline design has $n$ independent interpolation values.

Because no family is specified, the command fits a Gaussian identity-link generalized additive model:The $s_j$ are centered penalized regression splines with cubic regression bases; centering separates them from the intercept. The age curve in the original figure falls to a minimum near age 25, then rises markedly, particularly from about 40 onwards. Its effective degrees of freedom are $4.05$, reflecting curvature. The white-cell curve has effective degrees of freedom one and is nearly flat with a slight positive slope; the pointwise uncertainty bands are consistent with no substantial effect. These plots display centered partial mean contributions, not numbers of prescriptions by themselves.

A simpler mean model keeps the age regression spline and replaces the white-cell smooth by a linear term. A separate issue is that the response is a nonnegative count: a Poisson regression with logarithmic link function respects that support and positivity of the mean. A reasonable candidate isIf diagnostics show overdispersion, one can fitusing a negative binomial regression. Alternatively, if a Gaussian approximation is adequate, the minimal simplification is gam(npres ~ s(age, bs="cr") + wbc). These are candidates to compare by diagnostics and validation, rather than guaranteed improvements from the partial plots alone. Retain the curved age effect; consider a linear or removable white-cell effect, and assess an appropriate count family. Refit before interpreting the old curves on a new link scale.

There is a genuine syntax error in the printed command: random=list(...) must be a separate argument, following a comma rather than a plus. The intended call isWith the default Gaussian family, this generalized additive mixed model isHere $(b_{0j},b_{1j}{)}^T\stackrel{\mathrm{iid}}{\sim }{N}_2(0,D)$ across doctors, $D$ is an estimated covariance matrix, and the Gaussian errors have variance ${\sigma }^2$ and are independent of the random effects. The formula ~age includes a random intercept as well as a random slope; it does not force these two effects to be uncorrelated.

The random intercept allows different prescribing baselines, and the random slope allows doctor-specific linear modifications to the population age curve. Conditional on these effects the observations are independent under the model, whereas two patients of the same doctor have additional covariance $(1,a_{ij})D(1,a_{kj}{)}^T$. This accounts for clustered data and avoids treating doctor variation as independent patient-level noise. The population smooth and doctor-specific predictions are different targets. With only four doctors, estimates of the random-effect covariance can be imprecise; the Gaussian support issue for counts also remains. The new model accounts for doctor-specific baselines and age slopes.