Variational inference

Variational inference approximates a posterior distribution by minimizing a Kullback-Leibler divergence over a chosen family of probability distributions. For an unnormalized posterior density $h$ and a trial probability density function $q$, minimizing ${\mathbb{E}}_q[\log q-\log h]$ is equivalent to minimizing $\mathrm{KL}(q\Vert h/\int h)$.

A reparameterization gradient differentiates an expected value by representing the simulated random variable as a differentiable transformation of parameter-independent noise. For $X_{\eta }=g_{\eta }(\epsilon )$, the identity ${\nabla }_{\eta }\mathbb{E}[F_{\eta }(X_{\eta })]=\mathbb{E}[{\nabla }_{\eta }{F}_{\eta }(g_{\eta }(\epsilon ))]$ requires justified differentiation under the integral sign. For a normal distribution, use $X=m+e^{\rho }\epsilon$ with $\epsilon$ drawn from the standard normal distribution.

Markov chain variational inference optimizes an initial parametric probability distribution ${\nu }_{\eta }$ by minimizing $\mathrm{KL}(K^t{\nu }_{\eta }\Vert \pi )$ after a fixed number of Markov kernel updates. The best initial probability distribution can depend on $t$; Kullback-Leibler divergence contraction does not imply that optimization commutes with applying the Markov kernel.

Mean-field variational inference restricts the trial probability density function to $q(x)={\prod }_i{q}_i(x_i)$. With the other factors fixed, $q_j^{\ast }(x_j)\propto \exp ({\mathbb{E}}_{q_{-j}}\log \pi (x_j,X_{-j}))$, provided this expression has a finite positive normalization constant. Subtracting the objective at $q_j^{\ast }$ leaves $\mathrm{KL}(q_j\Vert q_j^{\ast })\ge 0$, proving the update is optimal when the terms are well defined.