Past exam of the mathematics course of the University of Cambridge / 2019 / iii / Paper 218 / 2 / d

Let $P_k$ be orthogonal projection onto the column space of $X_k$, and let $\mu$ be the true mean. ThenUp to a common constant,The true model has $p_k=p^{\ast }$ and ${\lambda }_k=0$; every other candidate has $p_k>p^{\ast }$ and ${\lambda }_k\ge 0$. Its excess expected AIC is ${\lambda }_k+p_k-p^{\ast }>0$, while its excess expected BIC is ${\lambda }_k+(\log n-1)(p_k-p^{\ast })>0$ when $n>e$. Thus either minimum expected criterion selects the true model.

With one true regressor and one additional candidate regressor, the reduction in residual sum of squares from fitting the larger nested model is $Z^2\sim {\chi }_1^2$. AIC chooses the wrong larger model exactly whenwhose probability is fixed and positive, independent of $n$. BIC chooses it whenFor $n>e^2$ this probability is strictly smaller than the AIC error probability, and it tends to zero as $n\to \infty$.