Daan Wierstra coauthored the 2014 work on stochastic backpropagation with Danilo Jimenez Rezende and Shakir Mohamed. The primary paper identifies all three with Google DeepMind in London. Their method jointly trains a generative model and a recognition network representing an approximate posterior.[ref]
Stochastic Backpropagation and Approximate Inference in Deep Generative Models by Danilo Jimenez Rezende and others. Title page; §§2 and4, generative and recognition models
An original scalar example isolates the shared inference-map parameter. For inputs , take a Gaussian sample , where has a standard normal distribution. The same controls both conditional means. Define the chosen toy lossIts derivative is , so at the gradient is . Holding a noise draw fixed permits the chain rule to pass through to the shared parameter, rather than optimizing a separate free mean for every input.
The paper's actual objective is a variational bound, not this squared-error illustration. Its Gaussian reparameterization provides a computable reparameterization gradient through random variables.[ref] The example explains one ingredient of that collaborative contribution without assigning the complete method to one coauthor.
New to topics? Read the docs here!