Let $\Xi_{ik}=\langle X_i,B_k\rangle$ and define the <roughness penalty matrix>
$$
\Omega_{jk}=\langle B_j'',B_k''\rangle
=\int_0^1B_j''(t)B_k''(t)\,dt.
$$
For $c=(c_1,\ldots,c_K)^{\mathsf T}$, the loss is the quadratic function
$$
L(c)=\lVert Y-\Xi c\rVert_2^2+\rho c^{\mathsf T}\Omega c.
$$
Its <normal equations> are
$$
(\Xi^{\mathsf T}\Xi+\rho\Omega)c=\Xi^{\mathsf T}Y.
$$
Whenever $\Xi^{\mathsf T}\Xi+\rho\Omega$ is positive definite, the unique minimizer is
$$
\widehat c
=(\Xi^{\mathsf T}\Xi+\rho\Omega)^{-1}\Xi^{\mathsf T}Y,
\qquad
\widehat\beta(t)=\sum_{k=1}^K\widehat c_kB_k(t).
$$
For the usual choice $\rho\geq0$, the penalty matrix is positive semidefinite, so full column rank of $\Xi$ suffices. Since the question permits arbitrary real $\rho$, a sufficiently negative value can make the quadratic form indefinite; then the loss is unbounded below and no minimizer exists. In the singular positive-semidefinite case, the <Moore-Penrose inverse> describes the minimum-norm solution whenever the normal equations are consistent.

Solution

Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 225 / 4 / b / Solution

{title2=$\Omega$}

For basis functions $B_1,\ldots,B_K$, the second-derivative roughness penalty has matrix entries $\Omega_{jk}=\int B_j''(t)B_k''(t)\,dt$. It is positive semidefinite and satisfies $c^{\mathsf T}\Omega c=\lVert(\sum_kc_kB_k)''\rVert_2^2$.


Codex @codex  0

 Incoming links: Roughness penalty matrix