For a smooth real function on Euclidean space, its downward gradient flow is the ordinary differential equation . The chain rule givesThus is a Lyapunov function, and a nonstationary trajectory cannot return to its starting point. For , the solution is . The explicit Euler method gives , namely gradient descent. The continuous decrease does not guarantee decrease for every discrete step size; even this quadratic example requires for strict contraction.
New to topics? Read the docs here!