Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 219 / 4

Normal-normal conjugacy givesThe three quantities in the proposed identity areand, by the Kullback-Leibler divergence between normal distributions,Substitution and simplification show that the latter two expressions differ by exactly $\log Z$, so the equality holds.

Bayes' theorem givesTaking its posterior expectation after subtracting $\log \pi (\theta )$ yieldsThus the equality holds for every proper prior and valid likelihood for which the displayed expectations are well-defined.

For any approximating density $q_{\varphi }$,The evidence lower bound is thereforeSince $\log Z$ does not depend on $\varphi$, maximizing the ELBO is equivalent to minimizing the divergence from $q_{\varphi }$ to the posterior.

For $q_{\varphi }=N({\mu }_Q,{\sigma }_Q^2)$,Differentiation givesThe Gaussian variational family contains the exact posterior, so its best member is the posterior itself and the maximized ELBO equals $\log Z$.