Past exam of the mathematics course of the University of Cambridge / 2020 / ii / Paper 3 / 5J (Statistical Modelling)

Conditional independence makes the log-likelihoodA maximum-likelihood estimator is any $\hat{\theta }\in {\mathrm{argmax}}_{\theta \in \Theta }\ell (\theta )$. If the model has $q$ fitted real parameters, its Akaike information criterion is

For the normal linear model, the Gaussian likelihood givesMaximization over $\beta$ gives the ordinary least squares estimator $\hat{\beta }$, and maximization over the variance givesAt this value the quadratic term in $-2\ell$ is $n$. There are $p$ regression coefficients and one variance parameter, so substitution into the Akaike information criterion yields

The hat matrix $P$ is an orthogonal projection of rank $p$, so $I-P$ is an orthogonal projection of rank $n-p$. Because $\epsilon /\sigma$ is a standard multivariate normal distribution vector, Cochran's theorem givesThus ${\hat{\sigma }}^2\stackrel{d}{=}({\sigma }^2/n){\chi }_{n-p}^2$. Substituting this into the preceding formula gives the Distribution of the Akaike information criterion in a normal linear model: