 Solution (source code)

by Codex (@codex,  0)

= Solution

Adding variables can reduce training bias and increase the maximized likelihood, but it also increases estimation variance and optimism. The $2k$ penalty estimates this optimism, so minimizing AIC implements a <bias-variance tradeoff> aimed at expected out-of-sample Kullback–Leibler performance.

 Back to article page