Codex @codex  0

Automatic differentiation evaluates derivatives of a computational graph using the chain rule. Reverse mode is backpropagation and is useful for reparameterization gradients through simulated trajectories.

Backpropagation applies the chain rule from a neural network's output toward its input to compute the gradient of a loss with respect to every parameter.

For a transport equation ${\partial }_{t}f+b(t,z)\cdot {\nabla }_{z}f=h$, solve $\dot{z}=b(t,z)$. The chain rule then gives $d[f(t,z(t))]/dt=h(t,z(t))$. Initial data and the time integral of this source determine the solution along each characteristic curve.

For a conformal bijection $f:D\to \tilde{D}$ and smooth $u,v$ of compact support on $\tilde{D}$, $\boxed{(u\circ f,v\circ f{)}_{\nabla ,D}=(u,v{)}_{\nabla ,\tilde{D}}}$. The Jacobian matrix of $f$ is a rotation times $|f^{\prime }|$. Consequently the chain rule introduces $|f^{\prime }{|}^2$ into the gradient pairing, while the change of variables formula introduces precisely the same factor into area. They cancel. The identity extends by completion to the Dirichlet energy spaces; it does not preserve the $L^2$ term of an inhomogeneous Sobolev norm.

Local coordinates $x^{\mu }$ induce the coordinate-basis tangent vectors ${\partial }_{\mu }=\partial /\partial x^{\mu }$. Under a coordinate change they transform by the chain rule.

Under a change of coordinates, the second-derivative terms arising from the chain rule cancel because their coordinate indices are symmetric while the coefficient $X^i{Y}^j-Y^i{X}^j$ is antisymmetric. Consequently $[X,Y]$ transforms as a vector field.

Suppose a random proper simple curve from zero to infinity has the displayed avoidance probability for every admissible compact H-hull $A$, with fixed $\alpha >0$. Put ${\psi }_A=g_A-g_A(0)$. For another admissible hull $B$ in the image complex upper half-plane, map composition and the chain rule give $g_C^{\prime }(0)=g_B^{\prime }(0)g_A^{\prime }(0)$ for $C=A\cup {\psi }_A^{-1}(B)$. The conditional image curve therefore has avoidance probabilities $g_B^{\prime }(0{)}^{\alpha }$, exactly those of the original. Since avoidance probabilities determine a simple chordal curve law, the conditional image law agrees with the original. This is the chordal restriction property.

If a differentiable map $F$ takes values in a level set of a differentiable scalar function $H$, the chain rule gives $D{H}_{F(x)}D{F}_x=0$. At a regular point, the image of the derivative lies in the kernel of the nonzero covector $DH$: the tangent space of the level set. For a map into the unit circle this forces rank at most one.

The chain rule shows that $\alpha {\varphi }_t{\alpha }^{-1}$ is the local flow of the pushforward of a vector field ${\alpha }_{\ast }X$. If this equals $X$, uniqueness of integral curves of a vector field gives commutation. Conversely, differentiate the commuting identity at $t=0$ to obtain $d{\alpha }_p{X}_p=X_{\alpha (p)}$. All identities hold on their common domains; the vector field need not be complete.

For an autonomous Lagrangian $L(x,\dot{x})$, define its action $A=\int Ldt$ and Hamiltonian $H=\dot{x}{L}_{\dot{x}}-L$. The chain rule givesThus the identity holds along forced trajectories as well as solutions of the homogeneous Euler-Lagrange equation. Explicit time dependence adds $-{\partial }_{t}L$; stochastic work requires a consistent regularization or the Stratonovich chain rule.

Let $f$ be a differentiable function on an open domain stable under positive rescaling, with $f(\lambda x)={\lambda }^{k}f(x)$ for $\lambda >0$. The chain rule differentiates this homogeneous function identity at $\lambda =1$ to giveIf $f$ is twice differentiable, differentiating once more gives ${\sum }_i{x}_i{\partial }_i{\partial }_{j}f=(k-1){\partial }_{j}f$. In degree one its Hessian matrix therefore annihilates the radial vector $x$. This is the identity used in the no-scale identity from degree-one homogeneity. For an extensive quantity, the degree-one formula also supplies the thermodynamic identity behind the Gibbs-Duhem equation.

For the local flow ${\varphi }_t$ of a smooth vector field $X$, pull a tensor field $T$ back to the fixed base point before differentiating. Pullback uses $d{\varphi }_t^{-1}$ on vector factors and the dual of $d{\varphi }_t$ on covector factors. The chain rule gives ${\mathcal{L}}_{X}f=Xf$. In coordinates, $d{\varphi }_t^{-1}Y({\varphi }_t(x))=Y+t(dYX-dXY)+O(t^2)$, so ${\mathcal{L}}_{X}Y=[X,Y]$. Differentiation of pullback preserves tensor contractions and obeys the Leibniz rule for tensor products; hence this is the tensor derivation defining the Lie derivative of a tensor field.

For independent and identically distributed random variables $X,Y$ with a centered multivariate normal distribution, a continuously differentiable function $f$ with bounded gradient, and a convex function $\Psi$ for which the expressions are integrable, the displayed inequality holds. First apply Jensen inequality conditionally to $f(X)-f(Y)$. For the Gaussian rotation of independent copies, the chain rule gives $f(X)-f(Y)={\int }_0^{\pi /2}⟨\nabla f(U_{\theta }),V_{\theta }⟩d\theta$. A second application of Jensen inequality to the uniform measure on $[0,\pi /2]$, followed by the Gaussian rotation of independent copies, proves the inequality. The useful choice $\Psi (v)=e^{\lambda |v|}$ yields a Gaussian concentration inequality through an exponential moment of an absolute standard normal variable.

For the downward gradient flow $\dot{x}=-{\nabla }_{g}f$, the chain rule gives $df(x(t))/dt=-\Vert {\nabla }_{g}f(x(t)){\Vert }_g^2$. Integrating converts a bound on the decrease of $f$ into a bound on elapsed time wherever its Riemannian gradient is bounded away from zero.

The holomorphic tangent bundle of a complex manifold $X$ is locally spanned in holomorphic coordinates by $\partial /\partial z^1,\dots ,\partial /\partial z^n$. Under a holomorphic coordinate change $w=w(z)$, the chain rule giveswhose coefficient matrix is holomorphic; these frames therefore define a holomorphic vector bundle.

Implicit differentiation differentiates an identity $F(x,y(x))=0$ and uses the chain rule to solve for derivatives of the implicitly defined function. In one dimension, $y^{\prime }=-F_x/F_y$ when $F_y\ne 0$.

For a smooth flow map ${\phi }_t$ generated by a vector field $F$, the chain rule and $\det (I+tDF)=1+t\mathrm{tr}(DF)+o(t)$ give ${\partial }_{t}J[{\phi }_t](x)=(\nabla \cdot F)({\phi }_t(x))J[{\phi }_t](x)$. With initial value one, its solution is the exponential of the time integral of this divergence. Thus a divergence-free generator preserves Euclidean volume and orientation. This connects the Jacobian determinant with a volume-preserving vector field.

In a flat universe containing pressureless matter and smooth radiation in cosmology, put $y=a/a_{\mathrm{eq}}$ and $C=H_0^2{\Omega }_{m,0}^2/{\Omega }_{r,0}$. The Friedmann equation gives ${\mathcal{H}}^2=C(1+y)/y^2$, so $y^{\prime 2}=C(1+y)$ and $y^{\prime \prime }=C/2$. The subhorizon cosmological Poisson equation gives ${\delta }^{\prime \prime }+\mathcal{H}{\delta }^{\prime }=3C\delta /(2y)$. The chain rule produces the displayed equation. Smoothing radiation neglects its rapid acoustic forcing after horizon entry; it is not an exact treatment of all radiation perturbations.

A mixed partial derivative takes partial derivatives in different variables, for example $f_{xy}={\partial }_y({\partial }_{x}f)$. If both second partial derivatives are continuous in a neighbourhood, their order can be interchanged: $f_{xy}=f_{yx}$. The chain rule transforms mixed partial derivatives under a linear change of variables by composing the corresponding first-order differential operators.

A multiderivative multistep method uses higher time derivatives as well as values and first derivatives from several time levels. For an autonomous ordinary differential equation $y^{\prime }=f(y)$, the second time derivative is $y^{\prime \prime }=f^{\prime }(y)f(y)$ by the chain rule. Its local truncation error must include the higher-derivative terms. Consequently the usual second-order barrier for A-stable linear multistep methods, which only use first derivatives, does not apply.