Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2022/iii/paper-218/5/b/solution

Adding variables can reduce training bias and increase the maximized likelihood, but it also increases estimation variance and optimism. The penalty estimates this optimism, so minimizing AIC implements a bias-variance tradeoff aimed at expected out-of-sample Kullback–Leibler performance.

New to topics? Read the docs here!