= Solution
Write $h(\theta)=\log p(\mathcal D\mid\theta)+\log p(\theta)$. Choose an interior <maximum a posteriori estimate>, a globally maximizing <posterior mode>
$$
\mu\in\operatorname*{arg\,max}_{\theta}h(\theta),\qquad \nabla h(\mu)=0.
$$
Suppose $h$ is twice continuously differentiable near $\mu$ and its negative <Hessian matrix> $A=-\nabla^2h(\mu)$ is a <positive-definite matrix>. The second-order <Taylor expansion> is
$$
h(\theta)\simeq h(\mu)-\frac12(\theta-\mu)^TA(\theta-\mu).
$$
Exponentiating and normalizing this local approximation gives the <Laplace approximation>
$$
\boxed{p(\theta\mid\mathcal D)\simeq\mathcal N(\theta;\mu,\Sigma),\qquad \Sigma=A^{-1}=-\bigl[\nabla^2h(\mu)\bigr]^{-1}}.
$$
Thus the approximate <mean> is the <posterior mode>, and the approximate <covariance matrix> is the inverse local curvature. This <normal approximation> is useful when most <posterior probability> is concentrated around this mode. A boundary mode, singular curvature, or several substantial modes requires a modified approximation.
Back to article page
