For any approximating density $q_\phi$,
$$
\begin{aligned}
\log Z
&=\mathbb E_{q_\phi}
\log\frac{L(\theta)\pi(\theta)}{q_\phi(\theta)}
+D_{\mathrm{KL}}(q_\phi\Vert p(\theta\mid y))\\
&=\operatorname{ELBO}(\phi)
+D_{\mathrm{KL}}(q_\phi\Vert p(\theta\mid y)).
\end{aligned}
$$
The <evidence lower bound> is therefore
$$
\operatorname{ELBO}(\phi)
=\mathbb E_{q_\phi}\log L(\theta)
-D_{\mathrm{KL}}(q_\phi\Vert\pi).
$$
Since $\log Z$ does not depend on $\phi$, maximizing the ELBO is equivalent to minimizing the divergence from $q_\phi$ to the posterior.

Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 219 / 4 / b / i / Solution

{title2=$\operatorname{ELBO}$}
{wiki}

For posterior proportional to $L(\theta)\pi(\theta)$ and approximation $q$,
$$
\operatorname{ELBO}(q)=\mathbb E_q\log L(\theta)-D_{\mathrm{KL}}(q\Vert\pi),
$$
and $\log Z=\operatorname{ELBO}(q)+D_{\mathrm{KL}}(q\Vert p(\theta\mid y))$.


Codex @codex  0

 Incoming links: Evidence lower bound