Lipschitz gradient
= Lipschitz gradient
{title2=$\|\nabla f(x)-\nabla f(y)\|_2\leq\beta\|x-y\|_2$}
A <differentiable function> has a $\beta$-<Lipschitz gradient> on a set $D$ if its <gradient> satisfies <Lipschitz continuity> there:
$$
\|\nabla f(x)-\nabla f(y)\|_2\leq\beta\|x-y\|_2\qquad(x,y\in D),\quad \beta\geq0.
$$
All <norms> here are the <Euclidean norm>. The constant limits how fast the <gradient> changes; it is not a bound on the magnitude of the <gradient>. For $f(x)=(x_1^2+9x_2^2)/2$, the <gradient> is the <linear map> $x\mapsto\operatorname{diag}(1,9)x$. Its <operator norm> is $9$, so $\beta=9$ works globally and is sharp by taking $x-y=(0,1)^T$. The <gradient> itself is unbounded on the whole <Euclidean space>.
If $f$ has a continuous <Hessian matrix> on an open <convex set> $D$ and $\|\nabla^2f(z)\|_2\leq\beta$ throughout $D$, integrate that <linear map> along the <line segment> from $x$ to $y$:
$$
\nabla f(y)-\nabla f(x)=\int_0^1\nabla^2f(x+t(y-x))(y-x)\,dt.
$$
The <operator norm> bound proves the displayed <Lipschitz continuity>. Convexity of the domain keeps the entire <line segment> inside it. Bounding the <Hessian matrix> only at the endpoints is insufficient.