Shakir Mohamed

Shakir Mohamed coauthored the 2014 stochastic-backpropagation paper with Danilo Jimenez Rezende and Daan Wierstra. It develops gradient computations through stochastic variables for joint learning of generative and approximate-inference models.^[ref] He also coauthored the 2015 work on normalizing flows with Danilo Jimenez Rezende, constructing flexible approximate posterior distributions through invertible transformations.^[ref]

An original smooth example shows why a reparameterization gradient is a computational object. Let $Z_{\mu }=\mu +ϵ$ with standard-normal $ϵ$, and choose $J(\mu )=\mathbb{E}[\sin Z_{\mu }]$. The exact expectation and derivative areDifferentiating a sampled value while holding its noise fixed gives $\cos (\mu +ϵ)$. At $\mu =0$, a chosen draw $ϵ=0$ yields gradient $1$, while the exact expectation gradient is $e^{-1/2}\approx 0.60653$. Averaging the sampled gradients over the stated noise distribution recovers the expectation gradient in this smooth bounded example.

That calculation depends on the distribution and differentiability assumptions. It does not make a single random derivative equal to the derivative of an expectation, nor does it by itself establish an analogous rule for a discontinuous sampling map. The cited contributions concern explicit probabilistic models and estimators, not an unrestricted license to differentiate through any random procedure.