= Solution
Let $0<p(x)<\infty$, take $q_\lambda$ supported where $p(z,x)>0$, and assume the following <expectations> exist. By the definition of <Kullback-Leibler divergence> and <Bayes' theorem>,
$$
\begin{aligned}
D_{\rm KL}(q_\lambda\Vert p(\cdot\mid x))
&=\mathbb E_{q_\lambda}\left[\log q_\lambda(z)-\log\frac{p(z,x)}{p(x)}\right]\\
&=\log p(x)-\mathbb E_{q_\lambda}[\log p(z,x)-\log q_\lambda(z)].
\end{aligned}
$$
Therefore the <evidence lower bound> is
$$
\boxed{\mathcal L(\lambda)=\mathbb E_{q_\lambda}[\log p(z,x)-\log q_\lambda(z)]=\log p(x)-D_{\rm KL}(q_\lambda\Vert p(\cdot\mid x)).}
$$
Nonnegativity of <Kullback-Leibler divergence> proves $\mathcal L(\lambda)\leq\log p(x)$. The <marginal likelihood> $p(x)$ is constant with respect to the variational parameter $\lambda$, so maximizing $\mathcal L$ minimizes the stated <Kullback-Leibler divergence>.
The practical advantage is that evaluating <logarithms> of joint <probabilities> and $\log q_\lambda$ requires no evaluation of the usually intractable <marginal likelihood>. One can estimate $\mathcal L$ and its <gradient> with <Monte Carlo method>. Equality with the log <marginal likelihood> occurs only when $q_\lambda=p(\cdot\mid x)$ almost everywhere. Merely maximizing the <evidence lower bound> over a restricted family neither computes the exact <posterior distribution> nor proves that the maximum attains this equality.
Back to article page
