Past exam of the MSc in advanced computer science course of the University of Oxford / 2021 / Bayesian statistical probabilistic programming / 2

Variational inference replaces exact Bayesian inference by an optimization problem over a tractable family $q_{\lambda }(z)$. A guide program samples from this approximating family. Optimizing an evidence lower bound is equivalent to minimizing $D_{\mathrm{KL}}(q_{\lambda }\Vert p(\cdot \mid x))$. Its appeal is that a fitted approximation can be cheap to sample and may scale well when an exact posterior distribution is inaccessible. A restricted family, such as mean-field variational inference, can miss dependence or entire modes; a local optimum can add further error. More optimization cannot remove approximation error caused by an inadequate family.

Monte Carlo method inference instead estimates posterior expectations using simulated values or weighted values. Examples include rejection sampling, importance sampling and Markov chain Monte Carlo. Under their respective support and convergence assumptions, increasing computation can yield consistent estimates without choosing a finite-dimensional posterior family. Their uncertainty includes simulation error; an inefficient proposal or slow mixing time can make that error large. Markov chain Monte Carlo additionally requires convergence and dependence diagnostics.

The central trade-off is optimization within an approximate family versus simulation under a target probability distribution. It is not an absolute distinction between deterministic and random computation: variational inference often uses Monte Carlo method estimates of its gradient, while Monte Carlo method inference may use fitted proposals. Neither method automatically guarantees accurate uncertainty estimates or superiority at a fixed computational budget.

Let $0<p(x)<\infty$, take $q_{\lambda }$ supported where $p(z,x)>0$, and assume the following expectations exist. By the definition of Kullback-Leibler divergence and Bayes' theorem,Therefore the evidence lower bound isNonnegativity of Kullback-Leibler divergence proves $\mathcal{L}(\lambda )\le \log p(x)$. The marginal likelihood $p(x)$ is constant with respect to the variational parameter $\lambda$, so maximizing $\mathcal{L}$ minimizes the stated Kullback-Leibler divergence.

The practical advantage is that evaluating logarithms of joint probabilities and $\log q_{\lambda }$ requires no evaluation of the usually intractable marginal likelihood. One can estimate $\mathcal{L}$ and its gradient with Monte Carlo method. Equality with the log marginal likelihood occurs only when $q_{\lambda }=p(\cdot \mid x)$ almost everywhere. Merely maximizing the evidence lower bound over a restricted family neither computes the exact posterior distribution nor proves that the maximum attains this equality.

The discrete base factors in the displayed construction inherit their variational parameters. Accordingly write $s_{\lambda }(\epsilon )$, not a parameter-independent $s(\epsilon )$, for the joint base probability mass function or probability density function. The continuous noise factors can be parameter independent while the discrete factors are $\lambda$ dependent. Calling the entire base law parameterless while retaining its nonzero score function would be inconsistent.

Let $t_{\lambda }$ push the base law forward to $q_{\lambda }$, and defineUse a fixed base measure $\nu$, with counting measure for discrete coordinates and Lebesgue measure for continuous coordinates. Assume locally parameter-independent support, differentiability and an integrable domination permitting differentiation under the integral sign. A valid reparameterization gradient givesApply the product rule before simplifying:This is the required combined score and pathwise gradient estimator. A sample from $s_{\lambda }$ and the integrand on the last line give an unbiased stochastic gradient under the stated regularity conditions. Its first term is a score-function gradient estimator contribution; the remaining terms differentiate the sampled objective along the continuous transformation. The same separation appears in the general stochastic computation graph construction.^[ref]

All derivatives inside the expectation hold $\epsilon$ fixed. They still include both explicit parameter dependence and dependence through $t_{\lambda }$. For a continuous coordinate $z=t_{\lambda }(\epsilon )$,The model joint density $p(z,x)$ is fixed as $\lambda$ changes, but its composition with $t_{\lambda }$ need not be constant. Discrete coordinates are not differentiated as real-valued paths; their changing probabilities enter through the score function. Moving supports or parameter-dependent branch boundaries require additional care and are not justified merely by formally interchanging a derivative and an integral.

The two limiting-case names in the printed bullets are reversed. Fully reparameterized continuous choices give a reparameterization gradient; non-reparameterized discrete choices give a score-function gradient estimator. Here are both reductions, including the extra term that vanishes only after taking an expectation.

When all choices are reparameterized with parameter-independent base noise, $s_{\lambda }=s$ and ${\nabla }_{\lambda }\log s=0$. The expression in part (c) becomesDifferentiating along the sampled transformation is precisely the pathwise gradient, also called the reparameterization gradient. There is no score function of the parameter-independent base law.

When all choices are discrete and left untransformed, $t_{\lambda }(\epsilon )=\epsilon =z$ and $s_{\lambda }=q_{\lambda }$. Set $S_{\lambda }(z)={\nabla }_{\lambda }\log q_{\lambda }(z)$ and $W_{\lambda }(z,x)=\log p(z,x)-\log q_{\lambda }(z)$. Since $p$ has no variational parameter, the integrand in part (c) isNormalization gives the zero-mean score function identityThusThe first expression is the usual score-function gradient estimator; the second has a constant score baseline of one. The two random integrands are generally different even though their expectations agree. A suitable constant score baseline can reduce variance, but one is not universally the optimal baseline.^[ref]

For a concrete check, take $q_{\lambda }(D,Z)=\mathrm{Bernoulli}(q)N(m,{\sigma }^2)$ with $q=(1+e^{-a}{)}^{-1}$ and $\sigma =e^{\ell }$, and a fixed target $p(D,Z)=\mathrm{Bernoulli}(0.4)N(D,1)$. Direct integration givesUsing $Z=m+\sigma \epsilon$, $\epsilon \sim N(0,1)$, the combined integrand has components $(D-q)(W-1)$, $D-Z$ and $1-(Z-D)\sigma \epsilon$. Their expectations are respectively $q(1-q)[\log (0.4/0.6)-\log (q/(1-q))+m-1/2]$, $q-m$ and $1-{\sigma }^2$, matching direct differentiation. Dropping the discrete score function term would incorrectly give zero for the first component. This calculation checks the combination without accepting the printed reversal.