For any approximating density

q_{ϕ}

lo g Z = E_{q_{ϕ}} lo g \frac{L ( θ ) π ( θ )}{q _{ϕ} ( θ )} + D_{KL} (q_{ϕ} ∥ p (θ ∣ y)) = ELBO (ϕ) + D_{KL} (q_{ϕ} ∥ p (θ ∣ y)) .

ELBO (ϕ) = E_{q_{ϕ}} lo g L (θ) - D_{KL} (q_{ϕ} ∥ π) .

Since

lo g Z

does not depend on

ϕ

, maximizing the ELBO is equivalent to minimizing the divergence from

q_{ϕ}

to the posterior.

ii

Solution

For

q_{ϕ} = N (μ_{Q}, σ_{Q}^{2})

ELBO (μ_{Q}, σ_{Q}^{2}) = - \frac{1}{2} lo g (2 π σ^{2}) - \frac{( y - μ _{Q} ) ^{2} + σ _{Q}^{2}}{2 σ ^{2}} - \frac{1}{2} [lo g \frac{τ ^{2}}{σ _{Q}^{2}} + \frac{σ _{Q}^{2} + μ _{Q}^{2}}{τ ^{2}} - 1] .

Differentiation gives

μ_{Q}^{*} = \frac{τ ^{2}}{σ ^{2} + τ ^{2}} y = θ, (σ_{Q}^{2})^{*} = \frac{σ ^{2} τ ^{2}}{σ ^{2} + τ ^{2}} = σ_{θ}^{2} .

The Gaussian variational family contains the exact posterior, so its best member is the posterior itself and the maximized ELBO equals

lo g Z

There are no discussions about this article yet.

There are currently no matching articles.