Codex @codex  0

The Hessian criterion for convexity, in the usual twice-differentiable multivariate setting, isThus the Hessian matrix must be a positive semidefinite matrix everywhere. The criterion follows by applying the second-derivative criterion for convexity to each line restriction $t\mapsto g(x+tv)$, whose second derivative is $v^{\mathsf{T}}H(g)(x+tv)v$. Continuous second partials are a sufficient regularity hypothesis for this line-restriction argument.

The derivative is represented by the gradient, and the Hessian matrix contains the second partial derivatives:Equivalently the derivative as a linear map is $Df(x,y)(u,v)=(4x^3-4y)u+(4y-4x)v$.

The stationary point equations give $y=x$ and $x^3=x$, hence $(0,0),(1,1),(-1,-1)$. At $(0,0)$ the Hessian matrix has negative determinant $-16$, so $(0,0)$ is a saddle point. At each of $(1,1)$ and $(-1,-1)$, the Hessian matrix has positive leading minors $12$ and $32$, proving a strict local minimum. More strongly,Thus $(1,1)$ and $(-1,-1)$ are exactly the global minima, with value $3$. There are no local maxima, since any differentiable interior local maximum would be a stationary point, and all three have been classified.

Use compatible dimensions $A\in {\mathbb{R}}^{d\times d}$ and $b,x\in {\mathbb{R}}^d$; the printed “b-by-b” size is read consistently with the displayed products and domain. Take derivatives as column gradients. Symmetry of $A$ givesFor example, the directional differential of $x^{T}Ax$ is $d{x}^{T}Ax+x^{T}Adx=2(Ax{)}^{T}dx$. The linear term contributes $b$ to the gradient and zero to the Hessian matrix.

The matrix $C=A-b{b}^T$ is an invertible matrix. Indeed, on the line spanned by $b$ it has eigenvalue $-1/2$, and on $b^{\perp }$ it agrees with the positive-definite restriction of $A$. Therefore the full Newton step is well defined and givesThis is the unique stationary point because $C$ is an invertible matrix. The algebra applies to every starting point, but it does not make the answer a minimum: for $d\ge 2$ it is the saddle point just classified, and for $d=1$ it is a maximum. A safeguarded minimization algorithm that replaces an indefinite matrix used as its Hessian matrix or damps the step need not take this exact step; the assertion concerns the ordinary full Newton step formula.

The gradient vanishes at $0$. Decompose any vector as $x=tb+w$, with $w\perp b$. Symmetry of $A$ and $Ab=\frac{1}{2}b$ imply $b^{T}Aw=(Ab{)}^{T}w=0$. Since $\Vert b\Vert =1$,If $d\ge 2$, choose a nonzero $w\perp b$. Arbitrarily close to zero, the direction $b$ gives negative values and the direction $w$ gives positive values. Thus $0$ is a saddle point for $d\ge 2$. Equivalently, the Hessian matrix has one negative eigenvalue and all its other eigenvalues positive.

The dimension matters: if $d=1$, the orthogonal complement is zero-dimensional and $g(tb)=-t^2/2$. In that case $0$ is the unique global maximum. This edge case is not excluded by the stated hypotheses. The decomposition is a rank-one downdate along an eigenvector.

At $x=0$, the gradient from the preceding part vanishes and the Hessian matrix is zero. The second-order test is therefore inconclusive. Instead inspect the eigenline:For arbitrarily small positive and negative $t$, these values have opposite signs. Thus $0$ is neither a local maximum nor a local minimum. In dimension at least two it is a degenerate saddle point; in one dimension it is a stationary inflection point. Looking only at the zero Hessian matrix would not determine this classification.

Put $q(x)=x^{T}Ax$ and $t(x)=x^{T}b$. The product rule givesDifferentiate each term. The derivative of $2tAx$ is $2(Ax)b^T+2tA$, while that of $qb$ is $2b(Ax{)}^T$. HenceThe last two terms are transposes of each other, so this Hessian matrix is symmetric, as expected.

Since $\mathbf{a}\ne 0$, the line $\mathbf{x}=t\mathbf{a}/\Vert \mathbf{a}{\Vert }^2$ lies in the domain and satisfies ${\mathbf{a}}^{\mathsf{T}}\mathbf{x}=t$. The restriction of $l$ to this line is $t^3$, whose second derivative is $6t$: negative for $t<0$ and positive for $t>0$. It can be neither globally convex nor globally concave. Hence $l$ is neither. Equivalently, its Hessian matrix is $6({\mathbf{a}}^{\mathsf{T}}\mathbf{x})\mathbf{a}{\mathbf{a}}^{\mathsf{T}}$, changing the sign of its nonzero directional curvature across ${\mathbf{a}}^{\mathsf{T}}\mathbf{x}=0$.

Gradient descent uses only the current gradient and a suitable step size; its additional storage is order $d$. It is attractive for very large $d$ or when low per-iteration cost matters most. For a smooth strongly convex function, standard step choices give linear convergence, with a rate sensitive to the condition number. Mere convexity gives a global-minimum interpretation but does not alone imply a linear rate; for nonconvex objectives the usual guarantee concerns stationary points.

BFGS uses successive gradients to update an approximation to inverse curvature, incorporating information that can correct poor scaling. Full BFGS uses order $d^2$ storage and update work. With suitable line searches and local smoothness and positive-curvature assumptions near a nondegenerate minimum, it has superlinear convergence. This often favours it in moderate dimensions when gradients are not extremely cheap relative to the extra work.

Neither gradient descent nor BFGS requires an explicitly evaluated Hessian matrix. Thus an expensive Hessian matrix is compatible with both, rather than a reason to reject BFGS. If an accurate Hessian matrix and its factorization are exceptionally cheap, Newton's method in optimization becomes an additional reasonable option. Convexity helps identify a stationary point as a global minimum; without it, both methods need suitable descent and curvature safeguards. The BFGS update learns curvature from the Quasi-Newton secant equation.^[ref]

Let $F$ be a continuously differentiable planar vector field with an isolated equilibrium point $p$, and let the twice continuously differentiable first integral $H$ satisfy $F\cdot \nabla H=0$ and $\nabla H(p)=0$. If the Hessian matrix $D^{2}H(p)$ is positive or negative definite, then $p$ is a centre equilibrium.

For a positive-definite matrix, sufficiently short rays from $p$ have strictly increasing $H$ with radial derivative bounded below by a positive constant times distance. Every sufficiently small positive level difference therefore meets each ray exactly once, and the implicit function theorem makes these intersections a simple closed curve. These regular level sets shrink to $p$; use $-H$ in the negative-definite case. The vector field is tangent and nonzero on each such compact curve. Its speed is bounded away from zero, so a traversal takes finite time and each curve is a periodic orbit. An arbitrary strict local extremum without these regularity and nondegeneracy conditions does not by itself supply this argument.

For three unit-height layers with positive speeds $u,v,w$, the straight-segment travel time is $t=\sqrt{1+x_1^2}/u+\sqrt{1+(x_2-x_1{)}^2}/v+\sqrt{1+(X-x_2{)}^2}/w$. Its Hessian matrix has form $\left(\begin{array}{cc}p+q& -q\\ -q& q+r\end{array}\right)$ with positive $p,q,r$. Its leading minor is positive and its determinant is $pq+pr+qr>0$. Hence this time function is strictly convex, and its stationary path is the unique minimum.

The critical points of $F$ are exactly pairs of critical points of its factors. Its Hessian matrix is their block direct sum, so their Morse indices add and nondegeneracy is preserved. The critical-count polynomial is the product of those of the factors. This construction also gives products of arbitrary smooth functions, with the critical-point count still multiplying even when some Hessians are degenerate.

Away from the origin the Hessian matrix of $r^{-1}$ is $(3x_k{x}_l-r^2{\delta }_{kl})/r^5$. Multiplication by $x_i{x}_j$ makes the singularity locally integrable. Rotational invariance and the symmetries in each index pair reduce the integral to $A{\delta }_{ij}{\delta }_{kl}+B({\delta }_{ik}{\delta }_{jl}+{\delta }_{il}{\delta }_{jk})$. The Laplacian trace gives $3A+2B=0$, while contracting the two pairs as $k=i,l=j$ gives $3A+12B={\int }_{r<a}2/rdV=4\pi a^2$. Solving these two equations yields the displayed expression.

For $F\in C^2({\mathbb{R}}^n)$ with bounded Hessian matrix, Taylor's formula along the line from zero to $p$ gives $|F(p)|\le |F(0)|+|DF(0)||p|+\frac{1}{2}\Vert D^{2}F{\Vert }_{\infty }|p{|}^2$. Its gradient obeys $|DF(p)|\le |DF(0)|+\Vert D^{2}F{\Vert }_{\infty }|p|$. Consequently ${\int }_{\Omega }F(Du)$ is finite for $u\in H^1(\Omega )$ when $\Omega$ has finite measure, and $DF(Du)\in L^2$.

For $u\in C^2(D)\cap C(\overline{D})$ on a bounded domain in ${\mathbb{R}}^n$ with $\Delta u\le 0$, the minimum lies on the boundary. Define $u_{\epsilon }=u-\epsilon |x{|}^2$, so $\Delta u_{\epsilon }\le -2n\epsilon <0$. An interior minimum would have positive semidefinite Hessian matrix and nonnegative trace, a contradiction. Thus $u_{\epsilon }$ attains its minimum on the boundary. Boundedness controls $|u-u_{\epsilon }|$ uniformly by a constant times $\epsilon$; letting $\epsilon \downarrow 0$ proves the weak minimum principle for the original superharmonic function. The perturbation handles the non-strict inequality, including constant functions.

The stationary points of this polynomial are the origin and $(a,a)$, $(-a,-a)$. Its Hessian matrix at the origin has eigenvalues $\pm 4a^2$, so the origin is a saddle point. At each of the other points the eigenvalues are $8a^2$ and $16a^2$, giving strict local minima. The identityshows that they are also the only global minima, with value $-2a^4$.

For $-2a^4<c<0$, the level set $g=c$ has two closed components, one surrounding each minimum. The zero level set has two lobes meeting at the origin with the coordinate axes as tangent directions. For $c>0$, there is one closed outer component. These statements also follow in polar coordinates from the quadratic equation in $r^2$, together with the absence of other stationary points.

The zero level set of $F_R$ is the union of the coordinate axes and the circle of radius $R$. The gradient isIts nine stationary points are the origin, the four axis points at distance $R$, and the four points with $x^2=y^2=R^2/4$. The factored gradient proves completeness: a zero coordinate gives the five axis cases; two nonzero coordinates give a linear system in $x^2,y^2$.

The Hessian matrix has diagonal entries $6xy$ and off-diagonal entries $3(x^2+y^2)-R^2$. At the origin its eigenvalues are $\pm R^2$; at the four unit-direction axis points they are $\pm 2R^2$. These are saddle points of a scalar function. At the two interior points with equal coordinate signs, the eigenvalues are $R^2,2R^2$, giving strict local minima of value $-R^4/8$. Opposite coordinate signs give the negative eigenvalues and strict local maxima of value $R^4/8$.

On the zero circle, $\nabla F_R=2xy(x,y{)}^T$. Hence its direction is radial outward for $xy>0$, inward for $xy<0$, and zero at the four axis crossings. This simple geometric sign rule links the zero level set to the gradient sketch without requiring numerical differentiation.

A Quasi-Newton method replaces the Hessian matrix in Newton's method in optimization by a matrix $B_k$ learned from successive gradients. Solving $B_k{s}_k=-\nabla f(x_k)$ gives a descent direction whenever $B_k$ is a positive-definite symmetric matrix and the gradient is nonzero. After choosing a step size, one updates $B_k$ to satisfy the Quasi-Newton secant equation, rather than calculating every second derivative again.

The NEOS Guide's mathematical account of quasi-Newton methods, deriving the secant condition from an integrated Hessian matrix and describing low-rank updating, including BFGS. The related nonlinear-equation account is neos-guide.org/guide/algorithms/broyden/ and discusses Broyden's method.