= Solution
<Variational inference> replaces exact <Bayesian inference> by an <optimization> problem over a tractable family $q_\lambda(z)$. A <guide program> samples from this approximating family. Optimizing an <evidence lower bound> is equivalent to minimizing $D_{\rm KL}(q_\lambda\Vert p(\cdot\mid x))$. Its appeal is that a fitted approximation can be cheap to sample and may scale well when an exact <posterior distribution> is inaccessible. A restricted family, such as <mean-field variational inference>, can miss dependence or entire modes; a local optimum can add further error. More optimization cannot remove approximation error caused by an inadequate family.
<Monte Carlo method> inference instead estimates posterior <expectations> using simulated values or weighted values. Examples include <rejection sampling>, <importance sampling> and <Markov chain Monte Carlo>. Under their respective support and convergence assumptions, increasing computation can yield consistent estimates without choosing a finite-dimensional posterior family. Their uncertainty includes simulation error; an inefficient proposal or slow <mixing time> can make that error large. <Markov chain Monte Carlo> additionally requires convergence and dependence diagnostics.
\b[The central trade-off is optimization within an approximate family versus simulation under a target <probability distribution>.] It is not an absolute distinction between deterministic and random computation: <variational inference> often uses <Monte Carlo method> estimates of its <gradient>, while <Monte Carlo method> inference may use fitted proposals. Neither method automatically guarantees accurate uncertainty estimates or superiority at a fixed computational budget.
Back to article page
