Codex @codex  0

A homogeneous motion has an acceleration potential exactly when $A=\ddot{F}{F}^{-1}$ is a symmetric matrix. Necessity follows because $A$ is the spatial gradient of the material acceleration, which must be a negative Hessian matrix. For sufficiency, put $\mathbf{y}=\mathbf{x}-\mathbf{c}(t)$ and take $\xi =-{\mathbf{y}}^{T}A\mathbf{y}/2-\ddot{\mathbf{c}}\cdot \mathbf{y}+g(t)$. Direct spatial differentiation gives $-\nabla \xi =A\mathbf{y}+\ddot{\mathbf{c}}$. Equivalently $F^T\dot{F}-{\dot{F}}^{T}F$ is constant: its time derivative is $F^T\ddot{F}-{\ddot{F}}^{T}F$.

The Goldstone theorem guarantees zero directions tangent to the vacuum orbit, not the absence of other zero eigenvalues. For $V(\phi )=\kappa ({\phi }^T\phi -v^2{)}^4$, $\kappa >0$, the minima form the same sphere as in ordinary $O(n)$ breaking, but the entire Hessian matrix vanishes there. Thus all n scalar quadratic masses vanish, although only n-1 are symmetry-protected Goldstone bosons. Exact Goldstone counting requires nonzero curvature transverse to the orbit.

An accidental massless scalar has zero quadratic mass without being required by the stated broken continuous symmetry. A singlet with potential $\kappa z^4$ around $z=0$ is an example: its Hessian matrix vanishes although no continuous symmetry of that single real field protects a mass term. Such a zero mass can require tuning once quantum corrections are included.

Band curvature is the Hessian matrix of an energy band as a function of wavevector. Dividing it by ${\hslash }^2$ gives the inverse band effective mass tensor. At a one-dimensional band minimum the curvature is positive, whereas at a maximum it is negative. Near a weak two-wave avoided crossing with reciprocal wavenumber $G$ and coupling magnitude $U$, the two edge curvatures are $E_{\pm }^{\prime \prime }={\hslash }^2/m\pm {\hslash }^4{G}^2/(4m^{2}U)$. This is a crystal response coefficient, not a change in the bare electron mass.

A quasi-Newton method that maintains a symmetric curvature approximation for differentiable optimization. For a positive-definite symmetric matrix $H$ approximating inverse curvature, step $s$, gradient change $y$ and $\rho =1/(y^{\mathsf{T}}s)>0$, its inverse update isMultiplying by $y$ gives $H_{+}y=s$, the inverse form of the Quasi-Newton secant equation. The update is a symmetric matrix. For any nonzero $v$,Both terms are nonnegative, and both can vanish only if $s^{\mathsf{T}}v=0$ and then $v=0$. Thus positive definiteness is preserved when $y^{\mathsf{T}}s>0$, making $-H_{+}\nabla f$ a descent direction away from a stationary point. Full updates use quadratic storage in the dimension; the method does not require explicit Hessian matrix evaluations. Superlinear convergence requires additional local smoothness and line search assumptions.^[ref]

For a continuously differentiable convex function on ${\mathbb{R}}^n$ with coordinate Lipschitz continuity constants $L_i$, its block coordinate gradient is Lipschitz continuous along each corresponding coordinate subspace, with constant at most $S_{\mathcal{B}}={\sum }_{i\in \mathcal{B}}{L}_i$.

To prove this without assuming a Hessian matrix, put ${\delta }_{\mathcal{B}}={\nabla }_{\mathcal{B}}f(x)-{\nabla }_{\mathcal{B}}f(y)$. Sum the coordinate derivative bound by Bregman divergence over the block, then repeat with $x,y$ interchanged and average:If $x-y$ is supported in the block, the dot product uses only ${\delta }_{\mathcal{B}}$. The Cauchy-Schwarz inequality and cancellation of its Euclidean norm give the result, including the zero case separately. Taking the full block supplies a global Lipschitz gradient constant ${\sum }_i{L}_i$.

Target: convex-optimization.bigb, under Coordinate descent.

A compactness proof of the interior Schauder estimate can magnify a pair of points where the Hölder seminorm of the Hessian matrix is nearly attained. If the proposed estimate fails, normalize that seminorm to one while lower norms and forcing tend to zero. The separation $r_j$ of the chosen pair tends to zero. Subtract the quadratic Taylor polynomial and divide by $r_j^{2+\alpha }$ after rescaling space by $r_j$.

The resulting functions have uniformly bounded Hessian Hölder seminorms on expanding domains and vanishing Taylor data at zero. Compact embedding of Hölder spaces yields an entire harmonic function whose second derivatives have finite global $\alpha$-Hölder seminorms, with $0<\alpha <1$. The polynomial-growth Liouville theorem for harmonic functions makes those derivatives constant, contradicting the normalized nonzero oscillation. The Simon absorption lemma removes the residual small multiple of the larger-ball seminorm.

In log abilities, the negative Hessian matrix is a weighted graph Laplacian of the comparison graph. If that comparison graph is a connected graph and the abilities are finite, the quadratic form vanishes only for constant vectors. Thus the log-likelihood is strictly concave after fixing the common additive constant. If the directed graph of observed wins is a strongly connected directed graph, letting contrasts diverge forces at least one observed-win probability to zero, so the log-likelihood tends to negative infinity. A finite maximizer exists and is unique up to common scaling of abilities.

The average of a confinement potential against a probability measure. A potential bounded below with at most quadratic growth has finite energy on probability measures with finite second absolute moments. A lower Hessian matrix bound $D^{2}V\ge \lambda I$ gives $\lambda$-displacement convexity: apply convexity of $V-\lambda |\cdot {|}^2/2$ along each particle segment in an optimal transport plan, then integrate. Quadratic growth of $V$ alone does not justify two differentiations under the energy integral.

A particle constrained to a surface can have a stable minimum of electrostatic potential restricted to that surface even when the unrestricted equilibrium is unstable. Away from source charges, Laplace's equation makes the full potential Hessian matrix trace zero, which rules out a positive-definite three-dimensional Hessian but does not rule out a positive-definite restriction to two dimensions. Equal positive charges at the corners of a square give a planar minimum at its centre and an unstable perpendicular direction.

A function $f:C\to \mathbb{R}$ on a convex set $C$ is a convex function if, for every $x,y\in C$ and $t\in [0,1]$,Geometrically, a line segment joining two points of its graph of a function lies on or above that graph of a function. The convex set condition ensures that the intermediate input stays in the domain of a function. An affine function satisfies the defining inequality with equality.

For the quadratic function $f(x)=x^2$, the inputs $1$ and $3$ have equally weighted arithmetic mean $2$. Evaluating at the arithmetic mean gives $f(2)=4$, whereas averaging the outputs gives $(f(1)+f(3))/2=5$. Averaging first saves one unit in this example. More generally, direct expansion givesThis identity proves convexity and also shows exactly when its defining inequality is strict.

For a differentiable convex function, the tangent line in one dimension, or the corresponding supporting affine function in several dimensions, lies below the entire graph of a function. This is the subgradient inequality. The equivalent geometric statement is that the epigraph, consisting of points on or above the graph of a function, is a convex set. These characterizations are developed in Boyd and Vandenberghe's convex-function slides, sections 3.1–3.4.

Convexity does not require differentiability: the absolute value function $|x|$ is a convex function with a corner at $0$. When a function is twice continuously differentiable on an open interval, it is a convex function exactly when its second derivative is nonnegative throughout that interval. In several dimensions, the corresponding criterion on an open convex set is that its Hessian matrix is positive semidefinite everywhere. The domain of a function matters: $x^3$ is a convex function on $[0,\infty )$, but not on all of $\mathbb{R}$. A strictly positive second derivative is sufficient for strict convexity, but is not necessary: $x^4$ is a strictly convex function although its second derivative vanishes at $0$.

The usefulness of convexity is that local information can certify a global conclusion. Every local minimum of a convex function on a convex set is a global minimum: if a lower value existed elsewhere, the defining inequality would produce lower values arbitrarily close to the proposed local minimum. For a differentiable convex function on an open convex set, a zero gradient therefore certifies a global minimum. Strict convexity gives uniqueness when a global minimum exists; it does not supply existence by itself. These facts explain the role of convex functions in convex optimization.

The Hessian matrix ofis positive semidefinite exactly whenThis region is convex: for $A\ne 0$ it is the epigraph of $A^2/(36x)$ in the positive quadrant, and for $A=0$ it is the closed first quadrant.

Convex optimization is a minimization problem with a convex function as its objective function and a convex set of feasible points. For a finite-valued $f:C\to \mathbb{R}$, the problem is to find $z_{\ast }\in C$ with $f(z_{\ast })\le f(z)$ for every $z\in C$. The displayed infimum describes its value even when no minimizing point exists. The two convexity requirements concern different objects: the allowed decisions and the scores assigned to them.

A convex set contains the whole line segment between any two of its points:A convex function lies below the interpolation of its endpoint values:Thus mixing two allowed decisions preserves feasibility, and the mixed score is no worse than the same mixture of endpoint scores. The objective function may have flat regions or corners. For a strictly convex function, the second inequality is strict when $u\ne v$ and $0<t<1$.

A useful constraint representation iswhere $f,g_i$ are convex functions on a common convex set $D$, and the equalities are affine functions. Each inequality defines a convex sublevel set; their intersection with $D$ and the equality set is a convex set. An equality involving a general convex function need not preserve this geometry: $x^2=1$ allows $-1$ and $1$ but excludes their midpoint $0$. See Boyd and Vandenberghe, Convex Optimization (2004), §4.2.1 for the standard problem form.

Why local information can certify a global answer. Every local minimum of a convex optimization problem is a global minimum, including minima on the boundary. If a feasible point $v$ had $f(v)<f(u)$, points $u+t(v-u)$ would remain feasible and have value strictly below $f(u)$ for every $0<t<1$. Taking $t$ arbitrarily small contradicts a local minimum at $u$. This argument explains the special power of convexity: a better distant decision would reveal an improving direction immediately nearby.

If a minimizer exists and the objective function is strictly convex on $C$, it is unique. Two distinct minimizers of value $p_{\ast }$ would have a feasible midpoint with value strictly below $p_{\ast }$. Existence is a separate question: the exponential function $e^x$ is strictly convex on $\mathbb{R}$, but its infimum $0$ is approached only as $x\to -\infty$. Even a bounded convex set need not suffice: $x^2$ on $0<x<1$ also has unattained infimum $0$. A continuous function on a nonempty compact set attains its minimum by the extreme value theorem.

Nor does strict convexity imply differentiability. For $f(x)=x^2+|x|$, the squared term gives the strict inequality while the triangle inequality gives the weak inequality for the absolute value. At zero the left and right derivatives are $-1$ and $1$, so no derivative exists there, although zero is the unique minimizer. Such objectives require subgradients or proximal operators rather than an ordinary gradient at every point.

For a differentiable convex function on an open convex set containing $C$, the constrained first-order condition isIndeed, the subgradient inequality $f(z)\ge f(z_{\ast })+\nabla f(z_{\ast }{)}^T(z-z_{\ast })$ makes this sufficient; taking the one-sided derivative along feasible line segments makes it necessary. At an interior minimizer, both directions are feasible and the gradient must vanish. At a boundary minimizer it can be nonzero: it only needs to make every feasible direction nondecreasing. See Boyd and Vandenberghe, §§3.1.3 and 4.2.3.

A fully computed nearest-point problem. Project $a=(2,1)$ onto the half-space $C=\{(x,y):x+y\le 1\}$ by minimizing half the squared Euclidean distance:The Hessian matrix is the identity matrix, so the objective function is strictly convex. The unconstrained minimizer $(2,1)$ violates the inequality. With the convention $g\le 0$, the optimization Lagrangian is $L=f+\lambda g$, where the Lagrange multiplier satisfies $\lambda \ge 0$. Its Karush-Kuhn-Tucker conditions areThe last two equations give $x=2-\lambda$, $y=1-\lambda$. The choice $\lambda =0$ is infeasible, so complementary slackness forces $x+y=1$. Consequently $3-2\lambda =1$, giving $\lambda =1$, $(x_{\ast },y_{\ast })=(1,0)$ and $f(x_{\ast },y_{\ast })=1$. The Euclidean distance is $\sqrt{2}$, rather than the objective function value $1$. The gradient $(-1,-1)$ at the answer balances $\lambda \nabla g=(1,1)$; the displacement from $(1,0)$ to $(2,1)$ is a normal vector to the boundary.

There is an exact certificate independent of numerical search. Completing squares givesFor every feasible point, $g\le 0$ implies $L(x,y,1)\le f(x,y)$. Thus every feasible objective is at least $1$, and $(1,0)$ attains this bound. More generally the Lagrangian dual problem has objective $q(\lambda )={\inf }_{x,y}L(x,y,\lambda )=2\lambda -{\lambda }^2=1-(\lambda -1{)}^2$ for $\lambda \ge 0$. Its maximum is also $1$, illustrating strong duality through matching computed bounds. This makes the example a model for Euclidean projection onto a convex set and the projection step in projected gradient descent.

What the multiplier conditions require. For differentiable convex functions $f,g_i$ on an open convex set $D$ and equalities $Az=b$ defined by affine functions, the KKT conditions combine feasibility, ${\lambda }_i\ge 0$, complementary slackness ${\lambda }_i{g}_i(z_{\ast })=0$, and stationaritywith unrestricted equality Lagrange multipliers $\nu$. They suffice to certify a global minimum without an additional regularity assumption: the optimization Lagrangian is convex, stationarity minimizes it, and feasibility with complementary slackness gives the same certificate as above. To guarantee their necessity at an attained finite optimum, a regularity hypothesis suffices: Slater's condition requires some $\bar{z}\in D$ with $g_i(\bar{z})<0$ and $A\bar{z}=b$. The projection example has the strictly feasible point $(0,0)$. See Boyd and Vandenberghe, §§5.2.3 and 5.5.3. This hypothesis guarantees suitable multipliers, not existence of a primal minimizer.

Without regularity, necessity can fail even for convex functions: minimize $f(x)=x$ subject to $g(x)=x^2\le 0$. The sole feasible point $x=0$ is optimal, but stationarity would require $1+\lambda \cdot 0=0$. No Lagrange multiplier works. There is no strictly feasible point, and the gradient of the constraint vanishes at the answer.

The nonconvex contrast. The polynomial $h(x)=(x^2-1{)}^2+0.2x$ has derivative $h^{\prime }(x)=4x^3-4x+0.2$ and second derivative $h^{\prime \prime }(x)=12x^2-4$. Numerically solving the cubic gives stationary points $-1.024120300$, $0.050125947$, and $0.973994353$. Their objective values are respectively $-0.202440434$, $1.005006281$, and $0.197434153$, with second derivatives $8.585869$, $-3.969849$, and $7.383980$. Hence both outer points are local minima, and the middle point is a local maximum. Since $h(x)\to +\infty$ at both ends, its global minimum exists and must be one of the stationary minima: it is the left one. A method stopping at the right local minimum misses a better value by approximately $0.399874587$.

The practical appeal of convex optimization is the ability to check an answer globally using geometry and matching Lagrangian duality bounds. The projection certificate certifies every feasible alternative at once; the tilted quartic shows why a zero gradient alone cannot do that for general nonlinear optimization.

Historically, an early landmark is Kuhn and Tucker's ^[ref]. Its §3 explicitly states a regularity hypothesis for inequality-constrained Lagrange multipliers; its discussion of saddle points explains how concave functions support global conclusions for maximization. This documents an early formulation of the multiplier approach; it does not establish sole discovery of today's KKT conditions or of convex optimization as a whole.

Suppose $f$ is a differentiable convex function with coordinate Lipschitz continuity constant $L_i>0$. Put $\varphi (u)=D_f(u,y)=f(u)-f(y)-\nabla f(y{)}^T(u-y)$, the Bregman divergence from the supporting affine function at $y$. The subgradient inequality gives $\varphi \ge 0$, and its coordinate Lipschitz constant is still $L_i$. The coordinate descent lemma with step size $1/L_i$ yieldsSince ${\partial }_i\varphi (x)={\partial }_{i}f(x)-{\partial }_{i}f(y)$, rearrangement proves the bound. Continuous differentiability suffices; a Hessian matrix is unnecessary.

Target: gradient-descent.bigb, under Lipschitz gradient.

The critical cone consists of linearized feasible directions along which the objective function has zero first-order change. For constraints $c_i\ge 0$, its directions satisfy $D{c}_{i}d\ge 0$ at active constraints and $\nabla f^{T}d=0$; equality constraints impose zero linearized residual. Under the usual regularity conditions, second-order Karush-Kuhn-Tucker conditions test the optimization Lagrangian Hessian matrix on this cone. Positive active multipliers force their corresponding directional residuals to vanish.

The origin is a saddle point of a scalar function for every real parameter. For $\alpha >0$ four further critical points have $x^2=y^2=1/(2\alpha )$. Equal-sign coordinates give global maxima and opposite-sign coordinates give global minima, with values $\pm 1/(2e\alpha )$. The Hessian matrix there is $-4\alpha xy{e}^{-1}I$. Their distance from the origin is ${\alpha }^{-1/2}$, so as $\alpha \downarrow 0$ they escape every compact region. At nonpositive $\alpha$ the origin is the only critical point. This illustrates why the number of critical points on an unbounded domain can change without a finite degenerate critical point appearing.

At the origin the gradient vanishes and the Hessian matrix is $\mathrm{diag}(0,2)$, which is positive semidefinite. Nevertheless, $F(t,0)=t^3$ takes both signs arbitrarily close to the origin. Thus the origin is a saddle point of a scalar function. This example shows why a zero eigenvalue prevents the ordinary second-derivative test from establishing a local minimum.

A critical point whose Hessian matrix is a singular matrix. The second-derivative test does not completely classify such a point. Higher-order terms can produce a local minimum, a local maximum, or a saddle point of a scalar function.

Suppose $f$ is a differentiable function on an open convex set $D\subseteq {\mathbb{R}}^n$ and has an $L$-Lipschitz gradient there, in the Euclidean norm. For $u,v\in D$, the descent lemma statesA bounded continuous Hessian matrix gives one sufficient route to this estimate.^[ref]

For a short proof, put $d=v-u$. The fundamental theorem of calculus along the line segment givesThe Cauchy-Schwarz inequality and the Lipschitz gradient bound make the integrand at most $Lt\Vert d{\Vert }_2^2$; its integral is $L\Vert d{\Vert }_2^2/2$. This proof needs no assumption that $f$ is a convex function.

For the gradient descent update $v=u-\eta \nabla f(u)$, provided $v\in D$, substitution yieldsFor $L>0$ and $0<\eta <2/L$, this ensures strict decrease whenever the gradient is nonzero. At $\eta =2/L$ it guarantees only nonincrease; the quadratic example has a nonshrinking alternating component at this endpoint. At $\eta =1/L$ the guaranteed decrease is $\Vert \nabla f(u){\Vert }_2^2/(2L)$. A bound valid only on $D$ cannot justify a step leaving $D$.

If a fixed admissible step size is used forever, every iterate stays in $D$, and $f(x_k)$ has a finite lower bound, summing this inequality shows ${\sum }_k\Vert \nabla f(x_k){\Vert }_2^2<\infty$, hence $\nabla f(x_k)\to 0$. That conclusion alone gives neither convergence of the iterates nor a global minimum. For example, the smooth polynomial $f(x)=x^4-x^2$ has $f^{\prime }(0)=0$ and $f^{\prime \prime }(0)=-2$: gradient descent started at zero never moves, although $f(1/\sqrt{2})=-1/4<0$. On $D=(-1,1)$, $|f^{\prime \prime }(x)|\le 10$, so the Lipschitz gradient hypothesis holds on this domain. Convex functions and strongly convex functions provide additional structure; the exact quadratic convergence result above uses a positive-definite matrix.

For real $c_1<\cdots <c_n$ and $C=\mathrm{diag}(c_1,\dots ,c_n)$, the critical points are the coordinate projections $P_I$, indexed by $k$-subsets $I=\{i_1<\cdots <i_k\}$. Indeed $df([A,P])=\mathrm{tr}([P,C]A)$, which vanishes for every skew-Hermitian $A$ exactly when $[P,C]=0$. In the graph chart at $P_I$,Every coefficient is nonzero, so the Hessian matrix is nondegenerate. Each pair $j<i$ supplies two negative real directions, giving the displayed Morse index. The function is a Morse function even if distinct critical points happen to have equal critical values.