Akaike information criterion

For a model with

k

fitted parameters and maximized likelihood

L (θ)

, the Akaike information criterion is

AIC = - 2 lo g L (θ) + 2 k .

Table of contents
- Distribution of the Akaike information criterion in a normal linear model Akaike information criterion
- Mallows Cp Akaike information criterion
  - Bias-variance decomposition for linear prediction Mallows Cp
  - Unbiased prediction-error identity for ordinary least squares Mallows Cp

Distribution of the Akaike information criterion in a normal linear model

For a full-rank

n \times p

normal linear model with error variance

σ^{2}

, the maximum-likelihood residual variance satisfies

σ^{2} = d \frac{σ ^{2}}{n} χ_{n - p}^{2} .

Consequently

AIC = d n lo g (χ_{n - p}^{2}) + n (lo g (2 π σ^{2} / n) + 1) + 2 (p + 1) .

Mallows Cp

For a normal linear model with known error variance

σ^{2}

, the scaled Akaike criterion differs by a model-independent constant from

C_{p} = ∥ Y - μ ∥^{2} + 2 p σ^{2} .

Bias-variance decomposition for linear prediction

If

Y, Y^{*}

are independent

N (μ, σ^{2} I_{n})

H

is a rank-

p

orthogonal projection, then

E ∥ H Y - Y^{*} ∥^{2} = ∥ (I - H) μ ∥^{2} + (n + p) σ^{2} .

The first term is squared approximation bias, while

p σ^{2}

is fitted-model variance and

n σ^{2}

is irreducible new-response noise.

Unbiased prediction-error identity for ordinary least squares

If

H

is the rank-

p

orthogonal projection onto a normal linear model's column space, then

E ∥ (I - H) Y ∥^{2} = ∥ (I - H) μ ∥^{2} + (n - p) σ^{2} .

Adding

2 p σ^{2}

gives

∥ (I - H) μ ∥^{2} + (n + p) σ^{2}

, so Mallows'

C_{p}

is unbiased for independent-copy prediction error even when the projection model is misspecified.

 Ancestors (7)

 Incoming links (37)

 Synonyms (1)

codex/aic

 View article source

 Discussion (0)

There are no discussions about this article yet.

 Articles by others on the same topic (1)

Akaike information criterion by

Wikipedia Bot  1

The Akaike Information Criterion (AIC) is a statistical measure used for model selection among a set of models. It is particularly useful when comparing different statistical models fitted to the same dataset. The AIC provides a means to evaluate how well a model explains the data, while also accounting for the complexity of the model to prevent overfitting.

 Read the full article

  See all articles in the same topic Create my own version