Chambolle–Pock algorithm Created 2026-10-06 Updated 2026-10-14
For a saddle point problem , one update ordering isThe proximal operators treat both nonsmooth convex terms, while the extrapolation couples the primal and dual variables. The primal objective is , with the convex conjugate. A dual-first ordering is obtained by exchanging roles; consistent indexing is needed when comparing the formulas.
Convergence of primal-dual hybrid gradient 2026-10-06
For proper lower semicontinuous convex functions on finite-dimensional Hilbert spaces, a nonempty saddle point set, and constant positive steps satisfying the displayed inequality, the primal-dual hybrid gradient method with extrapolation parameter one converges to a saddle point. The strict condition controls the bilinear coupling by the operator norm. It does not remove the need for existence of a saddle point. The fixed-step theorem and assumptions are recalled in Malitsky and Pock, A first-order primal-dual algorithm with linesearch, Section 1.
Convex optimization Created 2026-09-24 Updated 2026-10-13
Convex optimization is a minimization problem with a convex function as its objective function and a convex set of feasible points. For a finite-valued , the problem is to find with for every . The displayed infimum describes its value even when no minimizing point exists. The two convexity requirements concern different objects: the allowed decisions and the scores assigned to them.
A convex set contains the whole line segment between any two of its points:A convex function lies below the interpolation of its endpoint values:Thus mixing two allowed decisions preserves feasibility, and the mixed score is no worse than the same mixture of endpoint scores. The objective function may have flat regions or corners. For a strictly convex function, the second inequality is strict when and .
A useful constraint representation iswhere are convex functions on a common convex set , and the equalities are affine functions. Each inequality defines a convex sublevel set; their intersection with and the equality set is a convex set. An equality involving a general convex function need not preserve this geometry: allows and but excludes their midpoint . See Boyd and Vandenberghe, Convex Optimization (2004), §4.2.1 for the standard problem form.
Why local information can certify a global answer. Every local minimum of a convex optimization problem is a global minimum, including minima on the boundary. If a feasible point had , points would remain feasible and have value strictly below for every . Taking arbitrarily small contradicts a local minimum at . This argument explains the special power of convexity: a better distant decision would reveal an improving direction immediately nearby.
If a minimizer exists and the objective function is strictly convex on , it is unique. Two distinct minimizers of value would have a feasible midpoint with value strictly below . Existence is a separate question: the exponential function is strictly convex on , but its infimum is approached only as . Even a bounded convex set need not suffice: on also has unattained infimum . A continuous function on a nonempty compact set attains its minimum by the extreme value theorem.
Nor does strict convexity imply differentiability. For , the squared term gives the strict inequality while the triangle inequality gives the weak inequality for the absolute value. At zero the left and right derivatives are and , so no derivative exists there, although zero is the unique minimizer. Such objectives require subgradients or proximal operators rather than an ordinary gradient at every point.
For a differentiable convex function on an open convex set containing , the constrained first-order condition isIndeed, the subgradient inequality makes this sufficient; taking the one-sided derivative along feasible line segments makes it necessary. At an interior minimizer, both directions are feasible and the gradient must vanish. At a boundary minimizer it can be nonzero: it only needs to make every feasible direction nondecreasing. See Boyd and Vandenberghe, §§3.1.3 and 4.2.3.
A fully computed nearest-point problem. Project onto the half-space by minimizing half the squared Euclidean distance:The Hessian matrix is the identity matrix, so the objective function is strictly convex. The unconstrained minimizer violates the inequality. With the convention , the optimization Lagrangian is , where the Lagrange multiplier satisfies . Its Karush-Kuhn-Tucker conditions areThe last two equations give , . The choice is infeasible, so complementary slackness forces . Consequently , giving , and . The Euclidean distance is , rather than the objective function value . The gradient at the answer balances ; the displacement from to is a normal vector to the boundary.
There is an exact certificate independent of numerical search. Completing squares givesFor every feasible point, implies . Thus every feasible objective is at least , and attains this bound. More generally the Lagrangian dual problem has objective for . Its maximum is also , illustrating strong duality through matching computed bounds. This makes the example a model for Euclidean projection onto a convex set and the projection step in projected gradient descent.
What the multiplier conditions require. For differentiable convex functions on an open convex set and equalities defined by affine functions, the KKT conditions combine feasibility, , complementary slackness , and stationaritywith unrestricted equality Lagrange multipliers . They suffice to certify a global minimum without an additional regularity assumption: the optimization Lagrangian is convex, stationarity minimizes it, and feasibility with complementary slackness gives the same certificate as above. To guarantee their necessity at an attained finite optimum, a regularity hypothesis suffices: Slater's condition requires some with and . The projection example has the strictly feasible point . See Boyd and Vandenberghe, §§5.2.3 and 5.5.3. This hypothesis guarantees suitable multipliers, not existence of a primal minimizer.
Without regularity, necessity can fail even for convex functions: minimize subject to . The sole feasible point is optimal, but stationarity would require . No Lagrange multiplier works. There is no strictly feasible point, and the gradient of the constraint vanishes at the answer.
The nonconvex contrast. The polynomial has derivative and second derivative . Numerically solving the cubic gives stationary points , , and . Their objective values are respectively , , and , with second derivatives , , and . Hence both outer points are local minima, and the middle point is a local maximum. Since at both ends, its global minimum exists and must be one of the stationary minima: it is the left one. A method stopping at the right local minimum misses a better value by approximately .
A strictly convex quadratic has one minimum; a tilted quartic has two local minima with different values
. Details. The practical appeal of convex optimization is the ability to check an answer globally using geometry and matching Lagrangian duality bounds. The projection certificate certifies every feasible alternative at once; the tilted quartic shows why a zero gradient alone cannot do that for general nonlinear optimization.
Historically, an early landmark is Kuhn and Tucker's [ref]
https://digicoll.lib.berkeley.edu/record/112776/files/math_s2_article-35.pdf. Nonlinear Programming, Second Berkeley Symposium (1951), pp. 481–492
. Its §3 explicitly states a regularity hypothesis for inequality-constrained Lagrange multipliers; its discussion of saddle points explains how concave functions support global conclusions for maximization. This documents an early formulation of the multiplier approach; it does not establish sole discovery of today's KKT conditions or of convex optimization as a whole. Fast-slow matching for a calcium pulse Created 2026-10-31 Updated 2026-11-05
In a two-variable calcium-wave model, the fast front scale can reduce to the displayed dynamical system. Its equilibria are intersections of the kinetic nullcline and a line of slope . At a simple intersection with , the Jacobian determinant is ; a negative determinant identifies a saddle point. Matching a rising front requires a heteroclinic orbit from one saddle to another, selected by a shooting method in the speed . A falling connection must use the same speed. Between fronts the slow equation is . Recovery on the two outer rising branches requires the resting value to lie between the low landing value and the high takeoff value. The nullcline shape supplies this geometry but does not by itself prove existence of both connections.
Gaussian saddle and endpoint dominance by angle Created 2026-10-06 Updated 2026-11-05
For and , consider the Gaussian integral . Here, a saddle point contribution dominates for . The moving endpoint contribution in steepest descent dominates for . At the two endpoints have equal exponential rates and both are needed; for larger angles the fixed endpoint dominates. Saddle connectivity switches at , but that change is subdominant in the leading endpoint regime.
Gaussian saddle-pole transition Created 2026-10-05 Updated 2026-10-06
For a simple saddle point, a nearby pole coalesces with its contributing region when their separation is of order the inverse square root of the large parameter. A local quadratic phase coordinate changes the singular part to a Gaussian pole integral. Adding any residue crossed by the contour deformation gives a Faddeeva function rather than two separately singular approximations. The regular part of the transformed amplitude remains smaller by the ordinary saddle width.
Hénon-Heiles potential 2026-10-18
The potential energy in the Hénon-Heiles system has the following expression in polar coordinates:so it is invariant under rotation through . Solving gives a strict local minimum at the origin and three saddle points, at and . All three saddle points have value . Their common level set factors exactly:Thus this level set is the union of the three lines and . They bound an equilateral triangle containing the local minimum. The Hessian matrix is the identity at the origin; at each of the other three stationary points it has eigenvalues and . For energies , the central accessible region is bounded; above that value the three saddle passages allow paths from the centre to arbitrarily large radius. This follows directly from the radial potential: in each direction with , its radial maximum occurs at , with value .
Hessian determinant 2026-10-25
The determinant of the Hessian matrix. For a twice continuously differentiable function of two variables, it is . A negative value at a critical point implies a saddle point; a positive value combines with the sign of in the second-derivative test. A zero value is inconclusive.
Homoclinic orbit Created 2026-10-14 Updated 2026-11-05
For an equilibrium point , a homoclinic orbit is a nonconstant trajectory withAt a hyperbolic equilibrium point, the trajectory leaves a neighborhood of along its unstable manifold and returns along its stable manifold. It never reaches in finite time when local solution uniqueness holds. Its closure includes the equilibrium and can form a loop-shaped separatrix.
A completely explicit example iswith Hamiltonian function . Its derivative is zero along every trajectory, and the origin is a saddle point with eigenvalues . The curvelies on . Differentiating gives and , while the hyperbolic secant tends to zero at both time infinities. Thus it is a homoclinic orbit. Changing both signs gives the second loop.
Two exact homoclinic orbits of the double-well oscillator return to the same saddle at infinite time
. Details. Each loop bounds periodic orbits around one potential minimum. This is an integrable Hamiltonian system, so a homoclinic loop by itself does not establish chaos. A Shilnikov bifurcation and a simple-zero Melnikov energy-balance method calculation require additional dimensional, spectral or forcing hypotheses.
Morse lemma Created 2026-09-28 Updated 2026-11-05
Let be smooth, and let be a critical point whose Hessian is nondegenerate. There are smooth local coordinates , centered at , in whichThe integer is the Morse index. This is an exact local identity, rather than a quadratic approximation with a remainder.[ref]
To see the mechanism, apply the integral form of Taylor's theorem twice at the critical point, writing with smooth symmetric coefficients. Nondegeneracy allows successive smooth completion of squares; the resulting change of coordinates has derivative given by an invertible linear map, so the inverse function theorem makes it a valid coordinate system. Sylvester's law of inertia makes the number of negative squares independent of the coordinates.
For , the origin has index one and is a saddle point. For , the origin is critical but its Hessian is zero; no smooth coordinate change makes it a nondegenerate quadratic critical point. The normal form implies that every nondegenerate critical point is isolated, and supplies the local model for the Morse handle-attachment theorem.
Multivariable calculus Created 2026-09-24 Updated 2026-11-05
Multivariable calculus studies derivatives and integrals of functions with several independent variables. A temperature field , for example, has partial derivatives describing coordinate-wise changes, while its gradient gives the direction and rate of steepest increase. The Fréchet derivative combines directional changes into one linear map; its coordinate representation is the Jacobian matrix.
For , the gradient is and the directional derivative along a unit vector is . The Hessian matrix describes second-order variation and, under the hypotheses of the second-derivative test, distinguishes a local minimum from a saddle point. Constrained variation leads to Lagrange multipliers and the implicit function theorem.
Multiple integrals accumulate quantities over regions, as when integrating a mass density to obtain total mass. A change of variables formula uses the absolute Jacobian determinant to account for stretching of volume. For vector fields, the fundamental theorem for line integrals, Stokes theorem and divergence theorem relate interior derivatives to boundary measurements. They extend the endpoint difference of the ordinary fundamental theorem of calculus to circulation and flux.
Multivariable Newton iteration 2026-10-25
The finite-dimensional Newton root-finding iteration for a vector equation . At a point with invertible Jacobian matrix, solve the displayed linear system. For , its Jacobian matrix is the Hessian matrix, giving Newton's method in optimization. The update solves the affine first-order approximation ; it can converge to any nondegenerate critical point, including a saddle point.
Nondegenerate stationary points of a planar harmonic function Created 2026-10-18 Updated 2026-10-20
For a planar harmonic function, . Its Hessian matrix is therefore , with eigenvalues . At a stationary point with nonzero Hessian matrix determinant, both signs occur, so the point is a saddle point. A zero determinant requires separate treatment and is not covered by this nondegenerate test. The result is the local quadratic counterpart of the maximum principle for harmonic functions.
One-sided fluid-loaded membrane radiation 2026-10-14
For one acoustic half-space and vacuum on the other side, a harmonic point force gives displacement amplitude and fluid acoustic velocity potential amplitude . For time dependence , the outgoing acoustic square-root branch has for and for . The causal limit places the positive guided-mode pole above the inversion contour and the negative pole below. The saddle point generates cylindrical radiation, whereas guided residues decay normally to the membrane.
For a real symmetric invertible matrix , the full undamped Newton step for the displayed quadratic function sends every starting point to the unique stationary point:This follows because the gradient is affine and its first-order expansion is exact. If is a positive-definite matrix the result is the unique global minimum; a negative-definite matrix gives a global maximum, and an indefinite matrix gives a saddle point. Thus exact arrival does not by itself show that a minimization method has minimized anything. A line search, damping, or a positive-definite matrix replacing the Hessian matrix changes this exact one-step assertion.
Optimal truncation Created 2026-10-06 Updated 2026-10-14
Optimal truncation chooses how many terms to retain for a useful finite approximation. In a divergent asymptotic expansion, increasing the truncation order eventually makes the terms larger rather than smaller. Stopping near a smallest term often gives the best direct approximation, but the actual remainder must be checked. A small last term alone is not a universal error bound.[ref]
2.11 Remainder Terms; Stokes Phenomenon. NIST Digital Library of Mathematical Functions, §2.11(i), discussion of finite-parameter accuracy and independently checking remainders; §2.11(iii), exponentially improved expansions
For a divergent factorial series can approximate a convergent integral, decreases while and increases while . Thus the useful cutoff is near , rather than at arbitrarily large order. At , are the two smallest terms. Retaining terms givesThe actual error is about . Retaining twenty terms instead gives , with error about : more work produces a worse answer.
For this example the bound is explicit, so a requested tolerance can be tested before using a truncation. For a different integral, a nearby saddle point or simple pole can change the relevant remainder scale. This is why the smallest-term rule is a useful guide, rather than a replacement for remainder analysis.
The cutoff for factorial terms has . Better analysis can expose contributions smaller than every retained inverse power, linking optimal truncation to the Stokes phenomenon. Such contributions can be important even when many asymptotic coefficients agree. Directly summing ever more terms does not automatically recover them.
Parabola Created 2026-10-05 Updated 2026-10-18
A parabola is a plane curve whose points are equally distant from a focus and a directrix. In suitable coordinates it has equation , with . Its quadratic geometry can also describe a contour deformation through a saddle point.
For example, take a focus of a conic at and a directrix of a conic . Equality of squared Euclidean distances gives , hence . The point has distance to both the focus and the directrix. Its eccentricity of a conic is one. Unlike an ellipse, a parabola is unbounded and has only one matched focus-directrix description.
For , the signs needed for the phase portrait follow directly from the factorizations:The coordinate axes are invariant sets. On , the logistic differential equation directs positive states toward the boundary saddle point . On , directs positive states upward. At the three equilibria, use the local directions found in part (ii); interior sample trajectories near spiral clockwise inward.
The nullclines prescribe where arrows become vertical or horizontal; they are generally not themselves trajectories. The image samples the vector field and several solutions, rather than assuming every nullcline is an invariant set.
Intersect the two nullcline families. The axes yield and , and with yields . The possible remaining intersection would have and , requiring , whose discriminant is . Hence there are no further real equilibria.
- At it is . Both eigenvalues are positive, so the point is an unstable node.
- At it is . The eigenvalues have opposite signs, so this is a saddle point. Its stable direction is and its unstable direction can be taken as .
- At it is . Its characteristic polynomial is , so . This is an stable focus, or spiral sink.
All three are hyperbolic equilibria, so their classifications persist for the nonlinear system. Immediately to the right of on , the velocity is vertically downward, showing that the attracting spiral turns clockwise.
Since the exponential factor is positive, equilibria must have and . Thus . At either point the Jacobian matrix is . At its eigenvalues are , so this is a saddle point. At they are : the linearization has a centre equilibrium.
The requested first-order approximation follows by putting , : and . The linearized system conserves , giving the approximate ellipsesHowever, the printed nonlinear system has a weak repelling focus at , rather than a centre. A centre in the linearization need not be a nonlinear centre. Here that distinction can be proved explicitly, without relying on the numerical drift in the figure.
Put , , and . The weak focus of an exponential polynomial planar system has the local positive functionExpanding the vector field through cubic order and differentiating givesThe cubic correction cancels the cubic terms in the derivative of , and the quartic correction leaves the positive radial term . Since is positive and the angular velocity stays near , small nonzero trajectories spiral outward. Equivalently the reversed system has a strict local Lyapunov function and an asymptotically stable equilibrium. In particular, a periodic orbit sufficiently near the equilibrium is impossible, because would have to increase strictly and return to its starting value. This proves the failure of the printed centre assertion while retaining its correct linearized ellipse calculation.
The linearized ellipses hide the nonlinear outward spiral
. Original fourth-order Runge-Kutta integration of the printed vector field, compared with its linearization from the same initial point. The local polynomial calculation, rather than the plot, proves the weak instability. Details.




