Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 225 / 4

PutWith the $n$ by $K$ design matrix $\Xi =({\xi }_{ik})$, the scalar-on-function linear model becomesThe ordinary least squares estimator based on the truncated model isThus the omitted tail produces the conditional bias $({\Xi }^{\mathsf{T}}\Xi {)}^{-1}{\Xi }^{\mathsf{T}}r$.

LetThe supplied weak law of large numbers and noise limit giveFor a general fixed basis, $g$ need not vanish, so the retained coefficients are asymptotically biased and the estimator is inconsistent even for $c$. Moreover, with fixed $K$ it cannot recover the full slope $\beta$ when $\beta -{\beta }_K\ne 0$.

If the $B_k$ form the eigenbasis of $C_X$, thenThe retained functional principal component scores are therefore uncorrelated with every omitted score, so $g=0$. The least-squares estimates of $c_1,\dots ,c_K$ are consistent despite truncation. Fixed $K$ still estimates only the projection ${\beta }_K$; consistency for the complete function requires the truncation level to increase and the tail error to vanish.

Let ${\Xi }_{ik}=⟨X_i,B_k⟩$ and define the roughness penalty matrixFor $c=(c_1,\dots ,c_K{)}^{\mathsf{T}}$, the loss is the quadratic functionIts normal equations areWhenever ${\Xi }^{\mathsf{T}}\Xi +\rho \Omega$ is positive definite, the unique minimizer isFor the usual choice $\rho \ge 0$, the penalty matrix is positive semidefinite, so full column rank of $\Xi$ suffices. Since the question permits arbitrary real $\rho$, a sufficiently negative value can make the quadratic form indefinite; then the loss is unbounded below and no minimizer exists. In the singular positive-semidefinite case, the Moore-Penrose inverse describes the minimum-norm solution whenever the normal equations are consistent.