= Solution
For any approximating density $q_\phi$,
$$
\begin{aligned}
\log Z
&=\mathbb E_{q_\phi}
\log\frac{L(\theta)\pi(\theta)}{q_\phi(\theta)}
+D_{\mathrm{KL}}(q_\phi\Vert p(\theta\mid y))\\
&=\operatorname{ELBO}(\phi)
+D_{\mathrm{KL}}(q_\phi\Vert p(\theta\mid y)).
\end{aligned}
$$
The <evidence lower bound> is therefore
$$
\operatorname{ELBO}(\phi)
=\mathbb E_{q_\phi}\log L(\theta)
-D_{\mathrm{KL}}(q_\phi\Vert\pi).
$$
Since $\log Z$ does not depend on $\phi$, maximizing the ELBO is equivalent to minimizing the divergence from $q_\phi$ to the posterior.
Back to article page
