Automatic differentiation Created 2026-10-06 Updated 2026-11-05
Automatic differentiation evaluates derivatives by applying the chain rule to the elementary operations executed by a numerical program. It propagates local sensitivities through the calculation; it does not estimate them by choosing a finite increment as in numerical differentiation.
For , record , . A forward calculation carries both values and derivatives:It returns . More generally, seeding an input direction computes the product of a Jacobian matrix and a vector. A reverse calculation starts from an output sensitivity and accumulates contributions toward the inputs; for a scalar objective it computes the full gradient efficiently. This is the mechanism of backpropagation. [ref]
https://jmlr.org/papers/volume18/17-468/17-468.pdf. Automatic Differentiation in Machine Learning: a Survey, sections3.1–3.2, printed pp9–13
The chain-rule result is exact in exact arithmetic for the differentiable operations used; a floating-point implementation still has roundoff error. Branches and nondifferentiable operations require care: a derivative propagated through the executed branch need not be an ordinary derivative of the whole program at a branch boundary. The same chain-rule machinery can differentiate reparameterization gradients through simulated trajectories.
Combined score and pathwise gradient estimator Created 2026-10-25 Updated 2026-11-05
Under fixed-support regularity permitting differentiation under the integral sign, apply the product rule to . The first derivative term is and the second is . This proves the displayed identity. For an evidence lower bound, take , holding noise fixed in the inner derivatives. Parameter-independent continuous noise contributes a reparameterization gradient; untransformed discrete choices contribute a score-function gradient estimator. A mixed base law cannot be called parameter independent if its discrete factors retain variational parameters.[ref]
Daan Wierstra 2026-11-05
Daan Wierstra coauthored the 2014 work on stochastic backpropagation with Danilo Jimenez Rezende and Shakir Mohamed. The primary paper identifies all three with Google DeepMind in London. Their method jointly trains a generative model and a recognition network representing an approximate posterior.[ref]
Stochastic Backpropagation and Approximate Inference in Deep Generative Models by Danilo Jimenez Rezende and others. Title page; §§2 and4, generative and recognition models
An original scalar example isolates the shared inference-map parameter. For inputs , take a Gaussian sample , where has a standard normal distribution. The same controls both conditional means. Define the chosen toy lossIts derivative is , so at the gradient is . Holding a noise draw fixed permits the chain rule to pass through to the shared parameter, rather than optimizing a separate free mean for every input.
The paper's actual objective is a variational bound, not this squared-error illustration. Its Gaussian reparameterization provides a computable reparameterization gradient through random variables.[ref] The example explains one ingredient of that collaborative contribution without assigning the complete method to one coauthor.
Durk Kingma 2026-11-05
Durk Kingma is a computer scientist who publishes as Diederik P. Kingma. His personal profile describes research on machine learning and generative models, and records a 2017 doctorate at the University of Amsterdam.[ref]
Personal research homepage of Durk Kingma. Brief Bio: formal name, nickname and2017 doctorate
The institutional thesis front matter gives his full formal name as Diederik Pieter Kingma.[ref]Kingma and Max Welling coauthored Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Welling. A variational autoencoder couples a latent-variable model with an approximating posterior distribution that can be optimized through variational inference. Their reparameterization gradients represent a simulated random variable as a differentiable transformation of parameter-independent noise, allowing derivatives through the transformation.[ref]
Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Welling. §§2.3–2.4, equations4–5 and Gaussian construction
Kingma also coauthored the Adam paper with Jimmy Lei Ba.[ref]Adam: A Method for Stochastic Optimization by Diederik P. Kingma and Jimmy Lei Ba. Full byline and Algorithm1
An original scalar calculation makes the noise handling concrete. Use the Gaussian reparameterization , with having the standard normal distribution and . For , a one-draw reparameterization gradient with respect to isThe derivative holds the sampled fixed. This pathwise gradient is justified here by direct calculation; a general interchange of derivative and expected value requires differentiation under the integral sign conditions.
Noise coupling matters even for a central finite difference. If the evaluations at and share the same , the difference of the two squares gives exactly for every . This is a use of common random numbers. If instead the evaluations use independent standard normal distributions for their noise draws, the estimatorstill has expected value , but independence gives the variance of Gaussian-square finite-difference gradientsIndeed, a normal random variable of mean and variance has squared-value variance ; sum the two independent squared-value variances and divide by . Thus shrinking need not suppress simulation noise.
For chosen , and , both estimators have expected value 2, while their variances are 16 with shared noise and 2408 with independent noise. The example is a separate exact quadratic model; its purpose is to explain the role of keeping base noise fixed, not to reproduce a published variational-autoencoder benchmark.
Gaussian reparameterization 2026-11-05
Gaussian reparameterization represents a random variable with a normal distribution using a location, a scale and parameter-independent noise:Here has the standard normal distribution. The resulting has expected value and variance . The probability distribution is unchanged; the representation separates the random draw from the adjustable parameters. This makes reparameterization gradients available for differentiable expectations, including the Gaussian approximate posterior in a variational autoencoder.[ref]
Stochastic Backpropagation and Approximate Inference in Deep Generative Models by Danilo Jimenez Rezende and others. §3.2, Gaussian location-scale transformation and equation(10)
For a complete example, fix and considerHolding the same noise value fixed while differentiating givesAt , , the particular noise draw gives and derivative estimate , while the true derivative is . An individual sample need not equal the derivative; the equality is in expected value.
Parameterizing a positive scale as also gives . One can differentiate through the deterministic transformation while leaving the noise law fixed. For a general integrand, exchanging derivative and expectation still requires justified differentiation under the integral sign; the representation alone is not a proof of that exchange. Reusing noise when parameters change also connects the construction to common random numbers.
Max Welling 2026-11-05
Max Welling coauthored Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Welling with Durk Kingma. Their work combines a learned approximate posterior with a differentiable transformation of parameter-independent noise, permitting variational inference to use a reparameterization gradient. The original preprint was submitted in December 2013; its historical byline uses Diederik P. Kingma.[ref]
Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Welling. Primary byline, §§2.3–2.4 and §3
An original scalar calculation illustrates the computational mechanism. Let , where has a standard normal distribution and . For the chosen objective ,Holding one noise draw fixed while differentiating its squared transformed value instead gives the stochastic estimates and . At chosen , and , these are both , while the exact expectation gradients are and . Averaging over the noise, rather than treating one draw as the expectation, recovers the exact gradients in this example.
This small calculation isolates the differentiable sampling step. A variational autoencoder also specifies a probabilistic model and an evidence lower bound; a differentiable random transformation alone does not supply those components. The contribution is joint work, not a claim that one coauthor alone originated every part of the method.
Past exam of the mathematics course of the University of Cambridge 2017 iii Paper 216 4 c Solution Created 2026-10-03 Updated 2026-10-06
Parameterize the initial product normal distribution by , usingRun the given recursion with independent random variables with uniform distributions to obtain . Because the noise laws do not depend on , the chain rule and automatic differentiation give reparameterization gradients through the entire simulated trajectory. The variational objective is , where and .
One must account for the output differential entropy, not just optimize . Simulation and differentiability of do not themselves supply an evaluable . If retains the reversible Markov chain assumption from part (b), a concrete algorithm avoids a transition-density oracle using reversible density-ratio propagation. Write , with an evaluable unnormalized posterior density, and defineDetailed balance impliesFor example, integrate and replace by to obtain the first identity. This also proves absolute continuity of measures of with respect to when . An everywhere finite differentiable log-posterior has positive density, so an ordinary product normal distribution has this domination.
For each outer sample , independently run trajectories of length starting at , with endpoints . These are reverse trajectories because of detailed balance; the same recursion simulates them. Estimate byAverage over an outer batch. Compute it stably by a logarithmic sum of exponentials. Differentiate through both outer and reverse trajectories and through the explicit product-normal probability density function . Only the given state gradients of and , together with elementary normal-density derivatives, are needed. Apply stochastic gradient descent to and , refreshing the independent noise batches. The logarithmic scale parameters keep all variances positive.
This is nested Monte Carlo optimization of the exact objective in the limit of adequately increasing inner and outer sample sizes. For fixed , is generally biased, even though is an unbiased estimator. Finite logarithmic moments, suitable uniform integrability, and justified differentiation under the integral sign are needed for consistency and gradient interchange. Under those conditions increasing and controlling the optimization error yields a justified numerical approximation; nonconvexity prevents a generic guarantee of a global optimum.
There is a real assumption issue: part (c) redefines without explicitly restating reversibility. The algorithm above uses the reversible interpretation inherited from part (b). For an arbitrary differentiable recursion, one additionally needs access to output densities or a simulatable reverse kernel and regularity of the objective. Differentiability alone is insufficient: is differentiable and, with a standard-normal target, makes a point mass for , so every candidate has infinite Kullback-Leibler divergence. Thus a universal finite-objective algorithm cannot be justified solely from the literal differentiability assumptions. An invertible recursion with computable density transformations offers a different sufficient route, but invertibility is not stated here.
The discrete base factors in the displayed construction inherit their variational parameters. Accordingly write , not a parameter-independent , for the joint base probability mass function or probability density function. The continuous noise factors can be parameter independent while the discrete factors are dependent. Calling the entire base law parameterless while retaining its nonzero score function would be inconsistent.
Let push the base law forward to , and defineUse a fixed base measure , with counting measure for discrete coordinates and Lebesgue measure for continuous coordinates. Assume locally parameter-independent support, differentiability and an integrable domination permitting differentiation under the integral sign. A valid reparameterization gradient givesApply the product rule before simplifying:This is the required combined score and pathwise gradient estimator. A sample from and the integrand on the last line give an unbiased stochastic gradient under the stated regularity conditions. Its first term is a score-function gradient estimator contribution; the remaining terms differentiate the sampled objective along the continuous transformation. The same separation appears in the general stochastic computation graph construction.[ref]
All derivatives inside the expectation hold fixed. They still include both explicit parameter dependence and dependence through . For a continuous coordinate ,The model joint density is fixed as changes, but its composition with need not be constant. Discrete coordinates are not differentiated as real-valued paths; their changing probabilities enter through the score function. Moving supports or parameter-dependent branch boundaries require additional care and are not justified merely by formally interchanging a derivative and an integral.
The two limiting-case names in the printed bullets are reversed. Fully reparameterized continuous choices give a reparameterization gradient; non-reparameterized discrete choices give a score-function gradient estimator. Here are both reductions, including the extra term that vanishes only after taking an expectation.
When all choices are reparameterized with parameter-independent base noise, and . The expression in part (c) becomesDifferentiating along the sampled transformation is precisely the pathwise gradient, also called the reparameterization gradient. There is no score function of the parameter-independent base law.
When all choices are discrete and left untransformed, and . Set and . Since has no variational parameter, the integrand in part (c) isNormalization gives the zero-mean score function identityThusThe first expression is the usual score-function gradient estimator; the second has a constant score baseline of one. The two random integrands are generally different even though their expectations agree. A suitable constant score baseline can reduce variance, but one is not universally the optimal baseline.[ref]
Gradient Estimation Using Stochastic Computation Graphs by John Schulman and others. Section 4, Theorem 2 and footnote 3
For a concrete check, take with and , and a fixed target . Direct integration givesUsing , , the combined integrand has components , and . Their expectations are respectively , and , matching direct differentiation. Dropping the discrete score function term would incorrectly give zero for the first component. This calculation checks the combination without accepting the printed reversal.
Reparameterization gradient Created 2026-10-06 Updated 2026-10-25
A reparameterization gradient differentiates an expected value by representing the simulated random variable as a differentiable transformation of parameter-independent noise. For , the identity requires justified differentiation under the integral sign. For a normal distribution, use with drawn from the standard normal distribution.
Shakir Mohamed 2026-11-05
Shakir Mohamed coauthored the 2014 stochastic-backpropagation paper with Danilo Jimenez Rezende and Daan Wierstra. It develops gradient computations through stochastic variables for joint learning of generative and approximate-inference models.[ref]
Stochastic Backpropagation and Approximate Inference in Deep Generative Models by Danilo Jimenez Rezende and others. Primary byline; §§3–4
He also coauthored the 2015 work on normalizing flows with Danilo Jimenez Rezende, constructing flexible approximate posterior distributions through invertible transformations.[ref]An original smooth example shows why a reparameterization gradient is a computational object. Let with standard-normal , and choose . The exact expectation and derivative areDifferentiating a sampled value while holding its noise fixed gives . At , a chosen draw yields gradient , while the exact expectation gradient is . Averaging the sampled gradients over the stated noise distribution recovers the expectation gradient in this smooth bounded example.
That calculation depends on the distribution and differentiability assumptions. It does not make a single random derivative equal to the derivative of an expectation, nor does it by itself establish an analogous rule for a discontinuous sampling map. The cited contributions concern explicit probabilistic models and estimators, not an unrestricted license to differentiate through any random procedure.

