First order methods

First-order methods are a class of optimization algorithms that utilize first-order information, specifically the gradients, to find the minima (or maxima) of an objective function. These methods are widely used in various fields, including machine learning, statistics, and mathematical optimization, due to their efficiency and simplicity. ### Key Characteristics of First-Order Methods: 1. **Gradient Utilization**: First-order methods rely on the gradient (the first derivative) of the objective function to inform the search direction.