= Max Welling
{c}
<Max Welling> coauthored <Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Welling> with <Durk Kingma>. Their work combines a learned approximate posterior with a differentiable transformation of parameter-independent noise, permitting <variational inference> to use a <reparameterization gradient>. The original preprint was submitted in December 2013; its historical byline uses Diederik P. Kingma.<Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Welling>[Primary byline, §§2.3–2.4 and §3]{ref}
An original scalar calculation illustrates the computational mechanism. Let $Z=\mu+\sigma\epsilon$, where $\epsilon$ has a standard <normal distribution> and $\sigma>0$. For the chosen objective $J=\mathbb E[Z^2]$,
$$
J=\mu^2+\sigma^2,\qquad
\frac{\partial J}{\partial\mu}=2\mu,\qquad
\frac{\partial J}{\partial\sigma}=2\sigma.
$$
Holding one noise draw fixed while differentiating its squared transformed value instead gives the stochastic estimates $2Z$ and $2Z\epsilon$. At chosen $\mu=1$, $\sigma=2$ and $\epsilon=1$, these are both $6$, while the exact expectation gradients are $2$ and $4$. Averaging over the noise, rather than treating one draw as the expectation, recovers the exact gradients in this example.
This small calculation isolates the differentiable sampling step. A <variational autoencoder> also specifies a probabilistic model and an <evidence lower bound>; a differentiable random transformation alone does not supply those components. The contribution is joint work, not a claim that one coauthor alone originated every part of the method.
Back to article page
