Mean-vector prediction risk ($\mathbb{E}\Vert \hat{\mu }-\mu {\Vert }^2$)

The mean-vector prediction risk measures squared Euclidean norm error in estimating the deterministic response mean. For independent future noise with covariance matrix ${\sigma }^2{I}_n$, prediction of a new noisy response adds $n{\sigma }^2$ to this risk. A Gaussian orthogonal projection matrix $P$ of rank $k$ has risk $\Vert (I-P)\mu {\Vert }^2+k{\sigma }^2$.

For $Y\sim N_n(\mu ,{\sigma }^{2}I)$ and a fixed orthogonal projection matrix $P$ of rank $k$, the displayed expression is an unbiased estimator of the mean-vector prediction risk whenever ${\hat{\sigma }}^2$ is an unbiased estimator of ${\sigma }^2$. Independence of its two terms is unnecessary. Comparing fixed models yields the Mallows Cp penalty, but minimizing unbiased estimates does not preserve unbiasedness after selection.

For a normal distribution $N_n(\mu ,{\sigma }^{2}I)$ with unrestricted $\mu \in {\mathbb{R}}^n$ and unknown ${\sigma }^2$, an integrable data-only unbiased estimator of the mean-vector prediction risk of a fixed rank-$k$ orthogonal projection matrix exists exactly when $2k=n$. To prove necessity, randomize the mean by independent Gaussian variance $v$: conditioning adds $(n-k)v$ to the squared bias term, whereas the marginal variance identity would add $kv$. When $2k=n$, the residual sum of squares itself is unbiased.