= Daan Wierstra
{c}
{tag=Mathematics author}
<Daan Wierstra> coauthored the 2014 work on stochastic backpropagation with <Danilo Jimenez Rezende> and <Shakir Mohamed>. The primary paper identifies all three with Google DeepMind in London. Their method jointly trains a generative model and a <recognition network> representing an approximate posterior.<Stochastic Backpropagation and Approximate Inference in Deep Generative Models by Danilo Jimenez Rezende and others>[Title page; §§2 and4, generative and recognition models]{ref}
An original scalar example isolates the shared inference-map parameter. For inputs $x\in\{0,1\}$, take a Gaussian sample $Z_w(x)=wx+\epsilon$, where $\epsilon$ has a standard <normal distribution>. The same $w$ controls both conditional means. Define the chosen toy loss
$$
J(w)=\frac12\left(\mathbb E[Z_w(0)^2]+\mathbb E[(Z_w(1)-1)^2]\right)
=1+\frac12(w-1)^2.
$$
Its <derivative> is $w-1$, so at $w=0$ the gradient is $-1$. Holding a noise draw fixed permits the <chain rule> to pass through $Z_w(x)$ to the shared parameter, rather than optimizing a separate free mean for every input.
The paper's actual objective is a variational bound, not this squared-error illustration. Its <Gaussian reparameterization> provides a computable <reparameterization gradient> through random variables.<Stochastic Backpropagation and Approximate Inference in Deep Generative Models by Danilo Jimenez Rezende and others>[§3.2, equation10]{ref} The example explains one ingredient of that collaborative contribution without assigning the complete method to one coauthor.
Back to article page
