Past exam of the mathematics course of the University of Cambridge / 2003 / iii / Paper 41 / 6 / iii / Solution

The Laplace approximation evaluates an integral dominated by an interior maximum. Suppose $h$ has a unique interior maximizer $x_0\in {\mathbb{R}}^d$, negative-definite Hessian there, and sufficient smoothness and tail control. Put $H=-{\nabla }^{2}h(x_0)>0$. Taylor expansion under $x=x_0+u/\sqrt{n}$ gives a Gaussian leading integrand andThe leading factor follows from the multivariate Gaussian integral. Odd cubic terms integrate to zero over the limiting symmetric domain; fourth-order terms and squared cubic terms contribute to the first correction, assuming enough derivatives. If $g(x_0)=0$, a different leading term may be needed.

For a likelihood integral with prior density $\pi$, a common version is $\int e^{\ell (\theta )}\pi (\theta )d\theta \approx e^{\ell (\hat{\theta })}\pi (\hat{\theta })(2\pi {)}^{d/2}|j(\hat{\theta }){|}^{-1/2}$, because observed information is of order $n$. This is useful for posterior normalization, marginal likelihood and approximating integrated nuisance effects. A prior or integration measure is part of that construction; integrating nuisance parameters is different from merely profiling them. Boundary maxima, degenerate curvature, multiple modes or uncontrolled tails invalidate the simple one-mode formula.