Past exam of the advanced topics in statistical machine learning course of the University of Oxford / 2025 / 1 / c / i / Solution

Use the factorization $p(\theta ,\mathcal{D})=p(\theta \mid \mathcal{D})p(\mathcal{D})$ inside the evidence lower bound:HenceThe marginal likelihood is independent of the variational parameter $\varphi$, so maximizing the evidence lower bound is equivalent to minimizing the Kullback-Leibler divergence. Nonnegativity of the Kullback-Leibler divergence also gives $\mathcal{L}(\varphi )\le \log p(\mathcal{D})$.