If is an integer and , the characteristic function has continuous derivatives through order , with
Thus finite absolute moments justify recovering moments from derivatives.[ref]
For a short justification of differentiation under the integral sign, the difference quotient for the st derivative has integrand . Its absolute value is at most , since for real . Lower absolute moments are finite because . Apply the dominated convergence theorem successively, then use the same integrable bound to obtain continuity of each derivative.
For the Bernoulli distribution with parameter , direct differentiation gives and , agreeing with . Its variance is therefore . Existence and continuity of a characteristic function alone do not authorize these moment calculations: the standard Cauchy distribution, whose expected value is undefined, still has a characteristic function. The stated finite absolute moment hypothesis is a sufficient condition for the derivative formula, not an assumption built into the definition of a characteristic function.
Combined score and pathwise gradient estimator Created 2026-10-25 Updated 2026-11-05
Under fixed-support regularity permitting differentiation under the integral sign, apply the product rule to . The first derivative term is and the second is . This proves the displayed identity. For an evidence lower bound, take , holding noise fixed in the inner derivatives. Parameter-independent continuous noise contributes a reparameterization gradient; untransformed discrete choices contribute a score-function gradient estimator. A mixed base law cannot be called parameter independent if its discrete factors retain variational parameters.[ref]
Durk Kingma 2026-11-05
Durk Kingma is a computer scientist who publishes as Diederik P. Kingma. His personal profile describes research on machine learning and generative models, and records a 2017 doctorate at the University of Amsterdam.[ref] The institutional thesis front matter gives his full formal name as Diederik Pieter Kingma.[ref]
Figure 1.
Durk Kingma portrait linked from his personal research homepage
.
Kingma and Max Welling coauthored Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Welling. A variational autoencoder couples a latent-variable model with an approximating posterior distribution that can be optimized through variational inference. Their reparameterization gradients represent a simulated random variable as a differentiable transformation of parameter-independent noise, allowing derivatives through the transformation.[ref] Kingma also coauthored the Adam paper with Jimmy Lei Ba.[ref]
An original scalar calculation makes the noise handling concrete. Use the Gaussian reparameterization , with having the standard normal distribution and . For , a one-draw reparameterization gradient with respect to is
The derivative holds the sampled fixed. This pathwise gradient is justified here by direct calculation; a general interchange of derivative and expected value requires differentiation under the integral sign conditions.
Noise coupling matters even for a central finite difference. If the evaluations at and share the same , the difference of the two squares gives exactly for every . This is a use of common random numbers. If instead the evaluations use independent standard normal distributions for their noise draws, the estimator
still has expected value , but independence gives the variance of Gaussian-square finite-difference gradients
Indeed, a normal random variable of mean and variance has squared-value variance ; sum the two independent squared-value variances and divide by . Thus shrinking need not suppress simulation noise.
For chosen , and , both estimators have expected value 2, while their variances are 16 with shared noise and 2408 with independent noise. The example is a separate exact quadratic model; its purpose is to explain the role of keeping base noise fixed, not to reproduce a published variational-autoencoder benchmark.
Figure 2.
Variance of independent-noise and shared-noise quadratic gradient estimates
. Original analytic curves at mu = 1 and sigma = 2. Shared-noise central differences equal the pathwise derivative in this quadratic example; independent-noise variance grows as the finite-difference step shrinks. Details.
Energy conservation for the nonlinear Schrödinger equation Created 2026-09-24 Updated 2026-11-05
For the focusing nonlinear Schrödinger equation on ,
the energy functional is
For a sufficiently smooth solution with decay permitting differentiation under the integral sign and integration by parts, it is constant in time. Indeed,
The cancellation is the Hamiltonian conservation mechanism: the evolution rotates the energy gradient by multiplication by . The focusing sign makes the nonlinear contribution negative, so conserved energy alone need not bound the L2 norm of the gradient.[ref]
For less regular solutions the displayed computation is only formal. To inherit the identity, approximate by smooth solutions and justify convergence of both energy terms. Convergence in Sobolev space controls the nonlinear term through a suitable Sobolev embedding only in the applicable exponent range, or with additional integrability. An arbitrary weak solution must not be assumed to satisfy the identity solely because the smooth equation does.
For the angular-frequency Fourier transform and Schwartz seminorms , every output seminorm is bounded by finitely many input seminorms. To prove this, differentiation under the integral sign and integration by parts give
The generalized Leibniz rule, , and imply
This proves continuity of the Fourier transform on the Schwartz space and rapid decay of every transformed derivative.
The fundamental strip of a Mellin transform is the interior of the set of real parts for which
Viewed in the complex plane, it is a vertical strip ; either bound may be infinite, and the interior may be empty. Both endpoints of the improper integral must be checked. Convergence on a boundary line, whether absolute convergence or conditional convergence, requires a separate test. The Mellin transform is holomorphic throughout a nonempty fundamental strip.[ref]
A practical sufficient test uses Big O notation: if near zero and near infinity, with real , then comparison with elementary improper integrals gives
Thus guarantees absolute convergence. These upper bounds on need not locate the maximal Mellin fundamental strip; faster actual decay can widen it. The strip property itself follows by bounding the weight at any intermediate by the sum of two weights already known to be integrable. On a closed narrower strip, neighboring integrable weights also dominate the natural logarithm factors introduced by differentiation under the integral sign, making the transform a holomorphic function.
An exact two-endpoint example is
Here both bounds are sharp: at the zero-end absolute convergence test contains , and at the infinity-end test contains . The displayed rational function continues meromorphically outside the Mellin fundamental strip and has simple poles at and . Its other finite continued values outside the strip are values of a meromorphic continuation, not of a convergent defining integral. More general endpoint information leads to meromorphic continuation of a Mellin transform from an asymptotic expansion.
Figure 1.
Two endpoint conditions define the Mellin fundamental strip
. Left: the Mellin fundamental strip for below one and above one is . Crosses mark the two poles of the continued rational function, not whole lines of isolated singularities. Right: the formula and independently computed integrals agree on the real slice inside the strip. The curve is drawn only where the defining integral converges. Details.
Gaussian reparameterization represents a random variable with a normal distribution using a location, a scale and parameter-independent noise:
Here has the standard normal distribution. The resulting has expected value and variance . The probability distribution is unchanged; the representation separates the random draw from the adjustable parameters. This makes reparameterization gradients available for differentiable expectations, including the Gaussian approximate posterior in a variational autoencoder.[ref]
For a complete example, fix and consider
Holding the same noise value fixed while differentiating gives
At , , the particular noise draw gives and derivative estimate , while the true derivative is . An individual sample need not equal the derivative; the equality is in expected value.
Parameterizing a positive scale as also gives . One can differentiate through the deterministic transformation while leaving the noise law fixed. For a general integrand, exchanging derivative and expectation still requires justified differentiation under the integral sign; the representation alone is not a proof of that exchange. Reusing noise when parameters change also connects the construction to common random numbers.
Hadamard's lemma Created 2026-10-14 Updated 2026-10-18
On a star-shaped neighbourhood of , a smooth function satisfies
The fundamental theorem of calculus along the segment proves the identity, and differentiation under the integral sign proves smoothness of the coefficients.
Suppose is continuous at and, for some ,
For , its convolution with the heat kernel is smooth and solves the heat equation. On every compact set in , spatial or time derivatives of the heat kernel are bounded by the displayed integrable weight times a constant: their polynomial factors are absorbed by the strictly narrower Gaussian. This justifies differentiation under the integral sign.
Moreover, , so
Inside , continuity makes the difference uniformly small. Outside that interval, for , the inequality gives
The right side tends to zero, proving that the weighted tail contribution vanishes; the constant term is controlled by the vanishing Gaussian tail mass. Thus as . In particular, continuous data of polynomial growth satisfy this hypothesis, even when they are unbounded. This extends the Gaussian approximate identity argument beyond bounded data.
Let be continuous and suppose that the displayed improper integral converges separately at both infinite endpoints for some . Absolute convergence is not required. Then its heat-kernel solution exists, is smooth and solves the heat equation for , and tends to as at each fixed .
To prove the smoothing claim, take a primitive of . The assumed convergence makes bounded. For , integration by parts gives
The endpoint term vanishes. The coefficient of in the exponent of is . Its derivatives therefore have Gaussian decay on every compact subset of , justifying differentiation under the integral sign in the absolutely convergent right-hand expression. Since , it satisfies the heat equation.
For the initial limit, apply the same construction to and split at . Continuity controls the local part by its small oscillation times the unit kernel mass. On each remaining tail, is monotone for sufficiently small : its maximum lies at , approaching . If the relevant primitive has absolute bound , integration by parts bounds that tail by times the value of at its finite endpoint. For either endpoint , that value tends to zero like times a fixed constant. Both tails vanish, proving the limit. Separate endpoint convergence matters here; merely assuming a symmetric Cauchy principal value does not provide the bounded primitive used in this proof.
Leibniz integral rule 2026-10-18
For differentiable endpoints and a function with suitable continuous partial derivative in the parameter, the moving-endpoint rule is
To prove it locally, put for a fixed reference endpoint . The original integral is . Apply the chain rule, use the fundamental theorem of calculus for , and use differentiation under the integral sign for . The two boundary terms account for the moving endpoints, while the remaining integral accounts for variation of the integrand with the parameter. For improper integrals, convergence and a suitable domination condition must be verified before interchanging the derivative and integral.
The natural logarithm has the parameter-integral representation
For , the identity shows convergence: the absolute value is at most , which is integrable when . At , the apparent singularity is removable, with limiting value .
Write the left side as . On a compact parameter interval with , its differentiated integrand is bounded by the integrable function . The dominated convergence theorem justifies differentiation under the integral sign, giving . Since , the fundamental theorem of calculus gives . At , the original improper integral diverges as , and it also diverges for .
Logarithmic moments of a power on the unit interval Created 2026-10-18 Updated 2026-10-24
For real and integer , start from and differentiate times with respect to . Each derivative adds one factor , giving
To justify differentiation under the integral sign, restrict to a compact interval with lower endpoint . The derivatives are dominated by , which is integrable: turns its integral into . A power of is integrable against this decaying exponential. This also explains why is the convergence threshold.
For real , the displayed improper integral converges and is an even function of . For , differentiation under the integral sign and partial fraction decomposition give
For , use . Integrate the resulting rational functions and extend to by continuity. Integrating in and using gives for ; evenness gives the formula for negative parameters. Thus away from zero. At zero the one-sided derivatives are opposite, so is continuous but not differentiable. Substituting into the pointwise differentiated integrand would therefore be an invalid interchange.
For every real parameter ,
where . For , differentiation under the integral sign and a partial fraction decomposition give
Solving this first-order linear differential equation yields . To fix the constant, dominated convergence gives ; the last absolutely convergent integral changes sign under . Thus . Oddness in supplies the result for negative parameters. The initial value alone does not determine , since every solution tends to zero as .
Logarithmic sine integral 2026-10-18
For every real , define . The apparent endpoint singularity at is removable, with limit , and the bound ensures integrability. Differentiation under the integral sign is justified by the integrable bound on the parameter derivative. Thus
where and the last integral is a Laplace transform. Since , integrating with respect to gives . This obtains an integral with a logarithmic denominator by differentiating away that denominator, then restoring the parameter dependence with one elementary antiderivative.
Mean under one-sided exponential tilting Created 2026-10-14 Updated 2026-11-05
Suppose for all . Under the exponential tilting , is an integrable random variable whenever : its negative exponential tail is bounded, and its positive contribution is dominated by a larger positive exponential moment. Differentiation under the integral sign gives the displayed mean. At the endpoint , the positive part of is integrable, but the negative part need not be. The right derivative is valid as an extended value, possibly ; a finite mean at zero requires an extra negative-tail integrability assumption.
For real , define and . Smoothness of the integrands permits differentiation under the integral sign. Subtracting the two integrands gives , which is . This proves the displayed identity for . At , both sides extend continuously with value zero. The variable is the modulus; derivatives with respect to the parameter instead obey , with removable value at zero.
In the Laplace integral method for a differential equation, differentiate the integral three times with respect to , and replace by . Integration by parts gives
Thus the weight equation for this construction and its boundary condition are
which give
These are the conditions obtained by eliminating the transformed interior term and the endpoint term separately. As usual for this method, differentiation and integration by parts need convergence assumptions; the contours constructed next make every such step valid locally uniformly for all complex . A literal necessity claim for arbitrary weights and contours would additionally require uniqueness of the chosen representation by a Laplace integral; a null weight in a contour Laplace representation can otherwise be added without changing the solution.
Let
The quartic decay sectors for a contour integral contain the four rays . Along each ray, the quartic term is . For , choose to travel from infinity on ray to zero, and then from zero to infinity on ray . Set and define
On any compact set of values, the integrand and each of its derivatives are bounded by a constant times for some . These bounds are integrable at infinity, so differentiation under the integral sign is justified and the functions are entire. The endpoint term vanishes at both infinite ends. There is no extra term at zero because the two pieces join with the same integrand value and opposite endpoint orientations. Therefore all three functions solve the differential equation.
Figure 1.
Three independent Laplace integral contours joining rays in quartic decay sectors
. Details.
To prove independence, evaluate their first three initial data. For ,
If , these initial data give for . Define . Then
The coefficient matrix is a Vandermonde matrix at four distinct fourth roots of , and hence is invertible. All vanish. There is also a concrete reason for the earlier qualification about necessity. Deform the incoming ray of to angle and its outgoing ray to angle , within their respective quartic decay sectors. The quartic integral is unchanged and remains nonzero. On this deformed contour, the extra entire weight contributes zero for every : throughout the closing wedge, , so the large closing arc tends to zero and the Cauchy integral theorem gives a zero integral. Thus represents the same entire solution and has vanishing endpoint terms, although it does not satisfy . The canonical weight equation is the standard sufficient construction, rather than an unconditional uniqueness assertion about contour representations.
Thus the three contours give linearly independent nonzero solutions, and they form a fundamental system of solutions for this third-order linear equation. This proves independence directly instead of inferring it merely from the presence of three different contours.
For , the Schwarz integral on the unit disk, rescaled to radius , is
On each smaller closed disk , its kernel and all its derivatives in are uniformly bounded. Since the boundary values of the real harmonic function are continuous on this circle, differentiation under the integral sign proves that is holomorphic. Taking the real part gives the Poisson integral of those boundary values. It agrees with by uniqueness of the Dirichlet problem on the radius- disk. Hence is holomorphic and on . Its imaginary part is the harmonic conjugate normalized by .