 Home

Codex Wiki

Mathematics

Area of mathematics

Probability and statistics

Statistical inference

Bayesian statistics

{title2=$\operatorname{ELBO}$}
{wiki}

For <likelihood> $L$, prior density $\pi$, evidence $Z=\int L(\theta)\pi(\theta)\,d\theta$, and approximation density $q$, the <evidence lower bound> is
$$
\operatorname{ELBO}(q)=\mathbb E_q[\log L(\theta)]-D_{\rm KL}(q\Vert\pi).
$$
Whenever the terms are well-defined, substituting $p(\theta\mid y)=L(\theta)\pi(\theta)/Z$ into the <Kullback-Leibler divergence> gives $\log Z=\operatorname{ELBO}(q)+D_{\rm KL}(q\Vert p)$. Nonnegativity of the divergence proves the lower bound; equality holds when $q=p$ almost everywhere. <Variational inference> maximizes this objective over a tractable family, trading an exact posterior for an optimization problem.

Evidence lower bound

Combined score and pathwise gradient estimator

EM likelihood monotonicity

Kullback-Leibler divergence

Past exam of the advanced topics in statistical machine learning course of the University of Oxford / 2025 / 1 / c / iii / Solution

Past exam of the advanced topics in statistical machine learning course of the University of Oxford / 2025 / 1 / c / i / Solution

Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 216 / 4 / a / Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 219 / 4 / b / i / Solution

Past exam of the MSc in advanced computer science course of the University of Oxford / 2021 / Bayesian statistical probabilistic programming / 2 / a / Solution

Past exam of the MSc in advanced computer science course of the University of Oxford / 2021 / Bayesian statistical probabilistic programming / 2 / b / Solution

Relative entropy and coordinate dependence

Evidence lower bound ( $ELBO$ )

 Ancestors (6)

 Incoming links (11)

 Discussion (0) New discussion

 Articles by others on the same topic (0)

Evidence lower bound (ELBO)

 Ancestors (6)

 Incoming links (11)

 Discussion (0)  Subscribe (1) New discussion

 Articles by others on the same topic (0)

Evidence lower bound ( $ELBO$ )

 Discussion (0) New discussion