Bernstein-von Mises theorem

A Bernstein-von Mises theorem identifies a centered, scaled posterior distribution with the Gaussian law suggested by an efficient likelihood approximation. In an infinite-dimensional model, this conclusion requires control of prior-induced bias; a positive marginal prior density alone does not state enough assumptions.

A concrete calculation shows the issue. In the Gaussian sequence model, let $Y_k={\theta }_{0k}+n^{-1/2}{g}_k$, with ${\theta }_0\in {\ell }^2$, and assign independent priors ${\theta }_k\sim N(0,{\tau }_k^2)$ with every ${\tau }_k^2>0$ and ${\sum }_k{\tau }_k^2<\infty$. Put $a_{nk}=n{\tau }_k^2/(1+n{\tau }_k^2)$. For a nonzero coefficient vector $l\in {\ell }^2$, the linear functional $L(\theta )={\sum }_k{l}_k{\theta }_k$ has the exact posterior lawThe efficient observation statistic $L(Y)$ exists as a Gaussian noise pairing even though $Y$ itself is not in ${\ell }^2$. dominated convergence gives ${\sum }_k{l}_k^2{a}_{nk}\to \Vert l{\Vert }_2^2$. To center at $L(Y)$, however, one also needsThe remaining noise part of the scaled centering error has variance ${\sum }_k{l}_k^2(1-a_{nk}{)}^2\to 0$. Under this displayed bias condition, the conditional law of $\sqrt{n}[L(\theta )-L(Y)]$ converges in total variation distance, in sampling probability, to $N(0,\Vert l{\Vert }_2^2)$. Without bias control, convergence of posterior variances is insufficient. The calculation is specific to conjugate product Gaussian priors; general nonparametric results impose their own prior, truth and functional regularity hypotheses.^[ref]