Cauchy decrease bound

ID: cauchy-decrease-bound

Use the induced spectral norm for the symmetric matrix and interpret when . If , the Cauchy point gives a decrease at least . If , its step size satisfies and , so
Thus the numerical constant is independent of the model data. The bound links a nonzero gradient to a definite predicted decrease, including for an indefinite matrix .

New to topics? Read the docs here!