Past exam of the advanced topics in statistical machine learning course of the University of Oxford / 2025 / 1 / a / ii

Write $h(\theta )=\log p(\mathcal{D}\mid \theta )+\log p(\theta )$. Choose an interior maximum a posteriori estimate, a globally maximizing posterior modeSuppose $h$ is twice continuously differentiable near $\mu$ and its negative Hessian matrix $A=-{\nabla }^{2}h(\mu )$ is a positive-definite matrix. The second-order Taylor expansion isExponentiating and normalizing this local approximation gives the Laplace approximationThus the approximate mean is the posterior mode, and the approximate covariance matrix is the inverse local curvature. This normal approximation is useful when most posterior probability is concentrated around this mode. A boundary mode, singular curvature, or several substantial modes requires a modified approximation.