The displayed estimate, for all and real , defines a coordinate Lipschitz constant for the partial derivative in coordinate . It implies the coordinate descent lemmaby the fundamental theorem of calculus and integration of the Lipschitz continuity estimate. Subtracting an affine function leaves these constants unchanged.
The apparently weaker estimate along the singleton negative gradient direction is equivalent when every real step size is allowed: if the initial partial derivative is nonzero, any displacement can be written ; if it is zero but the final partial derivative is nonzero, reverse the endpoints; if both vanish the estimate is trivial.
Target: gradient-descent.bigb, under Lipschitz gradient. Articles by others on the same topic
There are currently no matching articles.