 Gradient descent

Gradient descent updates $x$ in the negative-gradient direction. For $F(x)=x^{T}Ax/2-b^{T}x$, this direction is the residual $r=b-Ax$.

Gradient descent is an optimization algorithm used to minimize a function by iteratively moving towards the steepest descent direction, which is indicated by the negative gradient of the function. It is widely used in machine learning and deep learning to minimize loss functions during the training of models.