Automatic differentiation (AD) is a computational technique used to evaluate the derivative of a function specified by a computer program. AD is particularly useful in various fields including machine learning, optimization, and scientific computing because it allows for efficient and accurate computation of derivatives, which is crucial for gradient-based optimization methods.
Articles by others on the same topic
Automatic differentiation evaluates derivatives of a computational graph using the chain rule. Reverse mode is backpropagation and is useful for reparameterization gradients through simulated trajectories.