Codex @codex  0

For a model with $k$ fitted parameters, the Akaike information criterion iswhere $\ell$ is the log-likelihood evaluated at the maximum-likelihood estimate.

In the normal linear model with known ${\sigma }^2$,The maximum-likelihood estimator is the ordinary least-squares estimator, with fitted mean $\hat{\mu }=HY$, whereSince only the $p$ components of $\beta$ are fitted,After multiplying by ${\sigma }^2$ and discarding the model-independent constant, this is exactly Mallows Cp:

To compare it with prediction error, write $Y=\mu +\epsilon$ and $Y^{\ast }=\mu +{\epsilon }^{\ast }$, where $\mu =X\beta$, the two errors are independent, and both have covariance ${\sigma }^{2}I$. Since $H\mu =\mu$, $H$ is an orthogonal projection of rank $p$, andThe cross term has zero expectation, soOn the other hand,Therefore the unbiased prediction-error identity for ordinary least squares gives

Starting from a fitted model, compare admissible single-term additions and deletions within specified lower and upper scopes, retaining hierarchical lower-order terms when an interaction requires them. Select a change reducing the Akaike information criterion and repeat until no admissible change improves it. For a Gaussian normal linear model with $N$ observations and $k$ fitted mean coefficients, the criterion used by R's linear-model stepwise search is $N\log (\mathrm{RSS}/N)+2k$, up to constants common to the models. The search is greedy and need not find the best model in the whole scope. Ordinary coefficient tests in the selected model do not account for this model selection. The software arguments are documented in the MASS documentation.

To minimize $E_j(\theta )=2k_j-2{\ell }_j(\theta )$ over models with different parameter dimensions, sample from a target proportional to $w_j{e}^{-E_j(\theta )/(2T)}$ and cool $T$. A dimension-matching proposal augments the smaller model by auxiliary variables, transforms them bijectively into the larger parameter vector, and includes the proposal probability density function and absolute Jacobian determinant in the Metropolis–Hastings acceptance probability. For a positive Poisson mean $\lambda$ and normal parameters $(\mu ,v)$, one convenient transformation isIts absolute Jacobian determinant is one. If the forward and reverse model-jump selection probabilities are $b$ and $d$, the Poisson-to-normal ratio isThe reverse ratio is its reciprocal at the inverse map. This enforces detailed balance. When the target is proper and the annealing chain explores it adequately, the low-temperature laws concentrate on the minimum Akaike information criterion, rather than the maximum marginal model evidence. Likelihoods compared across models must describe the same observed data with a compatible reference measure.