Akaike information criterion

For a model with $k$ fitted parameters and maximized likelihood $L(\hat{\theta })$, the Akaike information criterion is

The corrected Akaike information criterion adds a finite-sample penalty to the Akaike information criterion, where $k$ counts fitted parameters and $n>k+1$. The displayed correction is commonly used for small samples; its exact unbiasedness depends on the model family.

Starting from a fitted model, compare admissible single-term additions and deletions within specified lower and upper scopes, retaining hierarchical lower-order terms when an interaction requires them. Select a change reducing the Akaike information criterion and repeat until no admissible change improves it. For a Gaussian normal linear model with $N$ observations and $k$ fitted mean coefficients, the criterion used by R's linear-model stepwise search is $N\log (\mathrm{RSS}/N)+2k$, up to constants common to the models. The search is greedy and need not find the best model in the whole scope. Ordinary coefficient tests in the selected model do not account for this model selection. The software arguments are documented in the MASS documentation.

Deleting $q$ fitted mean parameters improves the Akaike information criterion exactly when the displayed condition holds, since the criterion difference is $n\log ({\mathrm{RSS}}_{\mathrm{reduced}}/{\mathrm{RSS}}_{\mathrm{full}})-2q$. For one parameter, the equivalent partial F-test statistic is below $(n-p)(e^{2/n}-1)$, where $p$ is the full mean-model dimension. This is not the usual fixed-level significance-test cutoff. Fits must use the same observations and response transformation.

For a full-rank $n\times p$ normal linear model with error variance ${\sigma }^2$, the maximum-likelihood residual variance satisfiesConsequently

For a normal linear model with known error variance ${\sigma }^2$, the scaled Akaike criterion differs by a model-independent constant from

If $Y,Y^{\ast }$ are independent $N(\mu ,{\sigma }^2{I}_n)$ vectors and $H$ is a rank-$p$ orthogonal projection, thenThe first term is squared approximation bias, while $p{\sigma }^2$ is fitted-model variance and $n{\sigma }^2$ is irreducible new-response noise.

If $H$ is the rank-$p$ orthogonal projection onto a normal linear model's column space, thenAdding $2p{\sigma }^2$ gives $\Vert (I-H)\mu {\Vert }^2+(n+p){\sigma }^2$, so Mallows' $C_p$ is unbiased for independent-copy prediction error even when the projection model is misspecified.