Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 205 / 5 / d / Solution

Because $x_{\ast }\sim N_p(0,I)$ is independent, conditional prediction risk equals $\mathbb{E}(\Vert {\hat{\beta }}_{\lambda }-{\beta }^0{\Vert }_2^2\mid X)$. With $\hat{\Sigma }=X^{T}X/n$ and $\lambda =n\ell$,The noise term has conditional mean zero and covariance $\frac{{\sigma }^2}{n}(\hat{\Sigma }+\ell I{)}^{-1}\hat{\Sigma }(\hat{\Sigma }+\ell I{)}^{-1}$. Taking squared norms proves