= Shakir Mohamed
{c}
{tag=Mathematics author}
<Shakir Mohamed> coauthored the 2014 stochastic-backpropagation paper with <Danilo Jimenez Rezende> and <Daan Wierstra>. It develops gradient computations through stochastic variables for joint learning of generative and approximate-inference models.<Stochastic Backpropagation and Approximate Inference in Deep Generative Models by Danilo Jimenez Rezende and others>[Primary byline; §§3–4]{ref} He also coauthored the 2015 work on <normalizing flows> with <Danilo Jimenez Rezende>, constructing flexible approximate posterior distributions through invertible transformations.<Variational Inference with Normalizing Flows by Danilo Jimenez Rezende and Shakir Mohamed>[Title page; §3]{ref}
An original smooth example shows why a <reparameterization gradient> is a computational object. Let $Z_\mu=\mu+\epsilon$ with standard-normal $\epsilon$, and choose $J(\mu)=\mathbb E[\sin Z_\mu]$. The exact expectation and derivative are
$$
J(\mu)=e^{-1/2}\sin\mu,\qquad J'(\mu)=e^{-1/2}\cos\mu.
$$
Differentiating a sampled value while holding its noise fixed gives $\cos(\mu+\epsilon)$. At $\mu=0$, a chosen draw $\epsilon=0$ yields gradient $1$, while the exact expectation gradient is $e^{-1/2}\approx0.60653$. Averaging the sampled gradients over the stated noise distribution recovers the expectation gradient in this smooth bounded example.
That calculation depends on the distribution and differentiability assumptions. It does not make a single random derivative equal to the derivative of an expectation, nor does it by itself establish an analogous rule for a discontinuous sampling map. The cited contributions concern explicit probabilistic models and estimators, not an unrestricted license to differentiate through any random procedure.
Back to article page
