Carrier wave 2026-10-14
A carrier wave is the rapid reference oscillation in a modulated wave. In the narrow-packet representation , choose near the centre of the frequency spectrum and on its dispersion relation. Its constant-wave phase reference moves at phase velocity . This reference need not coincide with a local maximum of the complete real wavefield when the wave envelope varies. Moving the reference shifts some slow wave phase into without changing .
A stationary point on the domain must be tested using allowed one-sided variations. For with , the boundary is a local minimum when and a local maximum when . At , the leading negative cubic term makes it a one-sided local maximum even though ; on an unrestricted real line it is instead a stationary inflection point.
Convex optimization Created 2026-09-24 Updated 2026-10-13
Convex optimization is a minimization problem with a convex function as its objective function and a convex set of feasible points. For a finite-valued , the problem is to find with for every . The displayed infimum describes its value even when no minimizing point exists. The two convexity requirements concern different objects: the allowed decisions and the scores assigned to them.
A convex set contains the whole line segment between any two of its points:A convex function lies below the interpolation of its endpoint values:Thus mixing two allowed decisions preserves feasibility, and the mixed score is no worse than the same mixture of endpoint scores. The objective function may have flat regions or corners. For a strictly convex function, the second inequality is strict when and .
A useful constraint representation iswhere are convex functions on a common convex set , and the equalities are affine functions. Each inequality defines a convex sublevel set; their intersection with and the equality set is a convex set. An equality involving a general convex function need not preserve this geometry: allows and but excludes their midpoint . See Boyd and Vandenberghe, Convex Optimization (2004), §4.2.1 for the standard problem form.
Why local information can certify a global answer. Every local minimum of a convex optimization problem is a global minimum, including minima on the boundary. If a feasible point had , points would remain feasible and have value strictly below for every . Taking arbitrarily small contradicts a local minimum at . This argument explains the special power of convexity: a better distant decision would reveal an improving direction immediately nearby.
If a minimizer exists and the objective function is strictly convex on , it is unique. Two distinct minimizers of value would have a feasible midpoint with value strictly below . Existence is a separate question: the exponential function is strictly convex on , but its infimum is approached only as . Even a bounded convex set need not suffice: on also has unattained infimum . A continuous function on a nonempty compact set attains its minimum by the extreme value theorem.
Nor does strict convexity imply differentiability. For , the squared term gives the strict inequality while the triangle inequality gives the weak inequality for the absolute value. At zero the left and right derivatives are and , so no derivative exists there, although zero is the unique minimizer. Such objectives require subgradients or proximal operators rather than an ordinary gradient at every point.
For a differentiable convex function on an open convex set containing , the constrained first-order condition isIndeed, the subgradient inequality makes this sufficient; taking the one-sided derivative along feasible line segments makes it necessary. At an interior minimizer, both directions are feasible and the gradient must vanish. At a boundary minimizer it can be nonzero: it only needs to make every feasible direction nondecreasing. See Boyd and Vandenberghe, §§3.1.3 and 4.2.3.
A fully computed nearest-point problem. Project onto the half-space by minimizing half the squared Euclidean distance:The Hessian matrix is the identity matrix, so the objective function is strictly convex. The unconstrained minimizer violates the inequality. With the convention , the optimization Lagrangian is , where the Lagrange multiplier satisfies . Its Karush-Kuhn-Tucker conditions areThe last two equations give , . The choice is infeasible, so complementary slackness forces . Consequently , giving , and . The Euclidean distance is , rather than the objective function value . The gradient at the answer balances ; the displacement from to is a normal vector to the boundary.
There is an exact certificate independent of numerical search. Completing squares givesFor every feasible point, implies . Thus every feasible objective is at least , and attains this bound. More generally the Lagrangian dual problem has objective for . Its maximum is also , illustrating strong duality through matching computed bounds. This makes the example a model for Euclidean projection onto a convex set and the projection step in projected gradient descent.
What the multiplier conditions require. For differentiable convex functions on an open convex set and equalities defined by affine functions, the KKT conditions combine feasibility, , complementary slackness , and stationaritywith unrestricted equality Lagrange multipliers . They suffice to certify a global minimum without an additional regularity assumption: the optimization Lagrangian is convex, stationarity minimizes it, and feasibility with complementary slackness gives the same certificate as above. To guarantee their necessity at an attained finite optimum, a regularity hypothesis suffices: Slater's condition requires some with and . The projection example has the strictly feasible point . See Boyd and Vandenberghe, §§5.2.3 and 5.5.3. This hypothesis guarantees suitable multipliers, not existence of a primal minimizer.
Without regularity, necessity can fail even for convex functions: minimize subject to . The sole feasible point is optimal, but stationarity would require . No Lagrange multiplier works. There is no strictly feasible point, and the gradient of the constraint vanishes at the answer.
The nonconvex contrast. The polynomial has derivative and second derivative . Numerically solving the cubic gives stationary points , , and . Their objective values are respectively , , and , with second derivatives , , and . Hence both outer points are local minima, and the middle point is a local maximum. Since at both ends, its global minimum exists and must be one of the stationary minima: it is the left one. A method stopping at the right local minimum misses a better value by approximately .
A strictly convex quadratic has one minimum; a tilted quartic has two local minima with different values
. Details. The practical appeal of convex optimization is the ability to check an answer globally using geometry and matching Lagrangian duality bounds. The projection certificate certifies every feasible alternative at once; the tilted quartic shows why a zero gradient alone cannot do that for general nonlinear optimization.
Historically, an early landmark is Kuhn and Tucker's [ref]
https://digicoll.lib.berkeley.edu/record/112776/files/math_s2_article-35.pdf. Nonlinear Programming, Second Berkeley Symposium (1951), pp. 481–492
. Its §3 explicitly states a regularity hypothesis for inequality-constrained Lagrange multipliers; its discussion of saddle points explains how concave functions support global conclusions for maximization. This documents an early formulation of the multiplier approach; it does not establish sole discovery of today's KKT conditions or of convex optimization as a whole. Critical point Created 2026-09-24 Updated 2026-10-13
For a smooth function on an open set , a critical point is an input where the gradient vanishes: . Equivalently, every first partial derivative is zero. Such a critical point is a stationary point; in one variable the condition is .
There is a broader convention in elementary one-variable calculus: a "critical number" is an interior input where the derivative is zero or does not exist, provided the function itself is defined there. Thus a corner can be a critical number without being a stationary point. Here the smooth meaning of critical point is used unless the broader convention is explicitly stated. [ref]
https://openstax.org/books/calculus-volume-1/pages/4-3-maxima-and-minima. Gilbert Strang and Edwin Herman, OpenStax Calculus Volume 1, §4.3: critical numbers
For the polynomial on the real line,The stationary points at are strict local minima and attain the global minimum; the stationary point at is a strict local maximum. One can check the local classifications from , which is positive at and negative at zero. The identity proves the global minimum directly. Since as , there is no global maximum on .
A zero derivative alone does not guarantee an extremum. The polynomial has but is strictly increasing through zero: every neighbourhood of zero contains both positive and negative values, so zero is neither a local minimum nor a local maximum. Conversely, the absolute value function has a strict local minimum and a unique global minimum at zero, but its left and right derivatives are and . It is not differentiable there, so zero is a critical number under the broader convention and is not a stationary point.
The usefulness of critical points is that they reduce the search for smooth interior extrema to equations. Every local minimum or local maximum of a differentiable function at an interior point has zero gradient; this condition is necessary, and shows that it is not sufficient. The striking feature is that all the first directional derivatives must vanish together, even though the surrounding values can form a valley, a peak, or a saddle point of a scalar function. [ref]
https://www.ocw.mit.edu/ans7870/18/18.013a/textbook/HTML/chapter14/section01.html. Daniel Kleitman, MIT Calculus with Applications, §14.1: necessary conditions for an interior extremum
Proof outline: at an interior local minimum in one variable, the difference quotient is nonnegative for small and nonpositive for small . If the derivative exists, the two sides have the same limit, forcing . Reverse the inequalities for a local maximum. In several variables, apply this argument along each coordinate line to obtain zero partial derivatives and hence zero gradient. At a domain boundary, both directions may not be available: on attains its global minimum at zero although for its smooth extension.
Critical point of a smooth map 2026-10-14
Imagine folding a sheet along its horizontal axis by the smooth map , . The inputs and have the same output. Along , horizontal motion still changes the output, but vertical motion produces no first-order change: . These inputs are critical points of a smooth map even though their differential is not the zero linear map. The missing direction in the target tangent space is what matters.
Precisely, let be a smooth map between smooth manifolds without boundary. An input is a critical point of a smooth map when its differentialis not surjective, equivalently when its rank of a linear map is less than . An input where it is surjective is a regular point; a map with no critical points of a smooth map is a submersion. The threshold is the target dimension, not . In manifold charts, compute the matrix rank of the Jacobian matrix; changes of charts preserve that matrix rank. [ref]
For the fold,Thus its set of critical points of a smooth map is the entire horizontal axis. Away from that axis, its differential can produce every infinitesimal target displacement. At the axis, it can produce only horizontal ones. This diagnoses where the two inverse branches in the picture merge.
For a scalar smooth function , the target dimension is one. A linear map to fails to be surjective exactly when it is zero, so here a critical point of a smooth map means . In Euclidean space this is precisely the familiar critical point condition : every first partial derivative vanishes. On a Riemannian manifold, it is equivalently for the chosen Riemannian metric; the Riemannian gradient depends on that metric, but whether it vanishes does not. See Loring W. Tu, An Introduction to Manifolds, Proposition 8.23 for the scalar coordinate criterion. The scalar smooth function has a critical point at zero without a local maximum or local minimum.
A source of lower dimension gives a different warning. For the smooth map , , the Jacobian matrix is the column , of matrix rank one at every input. Every input is a critical point of a smooth map, since the target dimension is two, even though this parametric curve has nonzero velocity everywhere and is an immersion. More generally, if , no differential can be surjective, so every input is critical. The terminology measures the ability to fill target directions, rather than whether the parametrized curve looks smooth.
EM likelihood monotonicity Created 2026-10-05 Updated 2026-10-14
For , the Jensen inequality givesThus any expectation-maximization algorithm M-step that increases cannot decrease the observed-data likelihood function.[ref]
For a short proof, assume common support and finite expressions, and keep the E-step conditional distribution fixed. The concavity of the logarithm and Jensen's inequality giveThis evidence lower bound equals . At , the ratio inside the expectation is the constant , so equality holds. During the M-step, the second term is constant; increasing raises a lower bound that touches the current log-likelihood. HenceIncreasing likelihood values alone do not establish convergence of the statistical parameters or attainment of a global maximum. In a differentiable finite mixture model, the gradient of equals the candidate-parameter gradient of evaluated at the current fit. Thus an interior fixed point of an exact M-step is a stationary point, which need not be a local maximum; the Identical-mean EM trap is an explicit counterexample. Small changes in log-likelihood can also occur before the means have settled, so inspecting both is useful.
Fermat's theorem on stationary points 2026-10-25
If a function is differentiable at an interior local minimum or local maximum, its derivative there is zero. At a local minimum , for small the difference quotient is nonnegative, while for small it is nonpositive. The one-sided limit of a function from each side equals , forcing and . Apply the same argument to at a local maximum.
The converse fails: has but no local minimum or local maximum at zero. The interior hypothesis is essential; a minimum attained only at an endpoint need not have derivative zero.
Global minimum Created 2026-09-24 Updated 2026-10-14
A real-valued function attains a global minimum at whenThe point is a global minimizer; is the minimum value. A global maximum reverses the inequality throughout the same domain of a function. Unlike a local minimum or local maximum, a global minimum or global maximum compares every permitted input, however distant. The domain of a function is part of the claim.
For on , completing the square giveswith equality exactly at . This proves that is the global minimum and that the two global minimizers are and . Neither is a strict global minimizer, because the other attains the same value; both are strict local minima. There is no global maximum on , whereas on the global maximum is , attained at zero.
Finding critical points supplies candidates rather than a global certificate. The squared expression above is beautiful because one inequality compares infinitely many inputs at once. A global minimum is always a local minimum under the relative-neighbourhood convention; the converse fails. For example, has a strict local minimum at , but as , so it has no global minimum on the real line.
Existence also matters: on has infimum zero and supremum one but attains neither a global minimum nor a global maximum. By the extreme value theorem, a continuous function on a nonempty closed interval attains both. To find those values, compare all interior critical numbers, including points where the derivative does not exist, and both endpoints. Checking only stationary points can miss a corner or an endpoint. [ref]
https://openstax.org/books/calculus-volume-1/pages/4-3-maxima-and-minima. Gilbert Strang and Edwin Herman, OpenStax Calculus Volume 1, §4.3: locating absolute extrema
Higher-order test for separated extrema Created 2026-10-07 Updated 2026-10-14
For , a degenerate Hessian matrix can be supplemented by the first nonzero term of each one-variable displacement. Two negative leading even powers give a strict local maximum, two positive ones a strict local minimum, and opposite signs give a saddle. Looking along the coordinate directions and bounding the separated terms avoids an inconclusive quadratic test.
For an explicit contrast, considerBoth have zero gradient and the same zero Hessian matrix at the origin:Nevertheless, is positive at every nonzero point, so the origin is a strict global minimum. For , the restrictions and for show that the origin is a saddle point of a scalar function. Identical quadratic information can conceal different geometry when higher-order terms are decisive.
Laplacian as departure from a local average 2026-10-18
The Laplacian detects whether a value lies below or above nearby values, to second order. For a scalar field whose partial derivatives through order four are continuous functions, take the arithmetic mean of four equally spaced neighbors:Applying Taylor's theorem in each coordinate cancels the first- and third-order terms between opposite neighbors. The two pairs leaveThis is an asymptotic expansion, rather than an equality for arbitrary . Rearranging gives the familiar central finite difference approximationThis finite difference replaces partial derivatives by five sampled values. The arithmetic mean lies above the center for a sufficiently small if the Laplacian is strictly positive there, and below if it is strictly negative. A zero Laplacian cancels this second-order discrepancy without requiring equality of individual values.
At , the bowl has center value zero and four neighbor values , so the discrepancy is exactly . The saddle has neighbor values , whose arithmetic mean is exactly zero. For the Gaussian function the discrepancy is , agreeing with .
A harmonic function need not equal this four-neighbor arithmetic mean for a finite . The polynomial has everywhere, but . Higher-order terms remain even when the second-order terms cancel.
At a local minimum, each coordinate second derivative is at least zero, hence the Laplacian is at least zero. At a local maximum the inequality reverses. The sum alone does not characterize an extremum: cancellation at a saddle point of a scalar function can also produce zero. The local-average interpretation explains why the heat equation smooths differences, and helps motivate the heat equation maximum principle.
Local extremum 2026-10-20
A local extremum of a real-valued function is either a local maximum or a local minimum, with comparisons restricted to a neighbourhood in its domain. A differentiable interior local extremum is a stationary point, but the converse can fail, as at zero shows.
Local maximum Created 2026-10-05 Updated 2026-10-14
A real-valued function , with , has a local maximum at if some satisfiesOnly a neighbourhood within the domain of a function is compared. Equality is allowed; a strict local maximum requires for every other point in that neighbourhood. A constant function therefore has a local maximum and a local minimum at every input, with neither strict. This is a local form of an extremum of a real-valued function.
For , zero is a strict local maximum because when . It is not a global maximum: values far away grow without bound. This distinction makes a local maximum useful for describing a nearby peak without claiming that it is the highest value anywhere.
Negating the function exchanges local maxima and local minima, including their strict versions. At a smooth interior local maximum, the critical point argument forces zero gradient; for a function twice continuously differentiable nearby, its Hessian matrix is negative semidefinite. These necessary conditions alone do not establish a local maximum.
Local maximum of Brownian motion 2026-10-06
The times of local maxima of a Brownian motion path are dense in the half-line almost surely, and every such local maximum is a strict local maximum. Density follows from nowhere monotonicity of Brownian motion and the extreme value theorem. Strictness follows by checking that maxima over separated rational compact time intervals are unequal; the increment across their gap supplies an independent nondegenerate normal distribution.
Mathematical optimization Created 2026-09-24 Updated 2026-10-13
Mathematical optimization turns a preference into a precise question: among the allowed decisions, which gives the best outcome? A model separates the objective function, which measures quality, from the optimization constraints, which determine the feasible set. This language unifies finding a cheap plan for a transportation problem, efficient scheduling, or an accurate model in linear regression. The separation is useful because changing a preference and changing what is possible are different modeling decisions. See Boyd and Vandenberghe, introductory optimization models.
For an objective function , a maximization problem asks for a feasible point satisfying for every ; a minimization problem reverses the inequality. This is a global extremum. A local maximum or local minimum compares only nearby feasible points, so finding one does not settle the global question.
Consider allocating ten units between two complementary inputs:The budget constraint forces a tradeoff: an extra unit of costs a unit of . The best allocation is , with objective function value . Without the budget constraint, gives : there is no global maximum. Constrained optimization can therefore have a finite answer even when the corresponding problem with fewer optimization constraints does not. Its beauty is that geometry, calculus, and an arithmetic-geometric mean inequality can certify the same answer.
Maximum and minimum 2026-10-13
Maxima and minima are extrema of a real-valued function. A global maximum of on its domain is a point with for every ; a global minimum reverses the inequality. A local extremum requires the inequality only in a neighbourhood of the point within . In constrained optimization, is the feasible set, so an extremum is tested relative to the constraints rather than in every ambient direction. Thus a constrained local maximum or local minimum can occur at a boundary point without being an interior critical point.
Nondegenerate critical point Created 2026-09-24 Updated 2026-10-14
A critical point of a real function with continuous second partial derivatives is nondegenerate when its Hessian matrix is an invertible matrix there, equivalently when none of its eigenvalues is zero. A positive-definite matrix gives a strict local minimum, a negative definite Hessian gives a strict local maximum, and opposite-sign eigenvalues give a saddle point of a scalar function. A nondegenerate critical point need not be an extremum: has Hessian matrix , which is an invertible matrix.
Such critical points are the local building blocks of Morse theory.
Nonisolated stationary point 2026-10-05
A stationary point is nonisolated if every neighbourhood contains other stationary points. For a twice differentiable function, a regular smooth curve of stationary points through a point forces its Hessian matrix to annihilate the curve's nonzero tangent: differentiate the vanishing gradient along the curve. Thus the Hessian matrix has a zero eigenvalue there, but that alone need not determine whether the point is a local minimum or local maximum.
The first instruction asks for maximization, whereas the later parts refer to minimization. For the literal maximization task, minimize the negative objective:
import numpy as np
from scipy import optimize as opt
def f(x):
return 1 + np.exp(-x*x/5) * np.sin(5*x)
maximum = opt.minimize(lambda z: -f(z[0]), x0=[-1.0])
print(maximum.x[0], f(maximum.x[0]))opt.minimize(lambda z: f(z[0]), x0=[-1.0]), which instead returns a local minimum near , . The optimizer's stopping criterion establishes local numerical convergence, not a certificate of a global optimum. Past exam of the mathematical methods II course of the University of Oxford 2021 1 ii Solution 2026-10-25
The product rule and chain rule giveThe exponential function is strictly positive, so there are exactly two stationary points, at . The derivative is negative outside these two values and positive between them. Thus the negative value is a strict local minimum and the positive value a strict local maximum.
Past exam of the mathematics course of the University of Cambridge 2001 ia Paper 3 9C Solution Created 2026-09-24 Updated 2026-10-14
At a critical point of a twice continuously differentiable real function, the Taylor expansion isThe Hessian matrix is real symmetric, so an orthogonal eigenbasis diagonalizes its quadratic form. If every eigenvalue is positive, that form is bounded below by a positive multiple of , which dominates the remainder and gives a strict local minimum. If every eigenvalue is negative, the same argument gives a strict local maximum. If both signs occur, displacement along the corresponding eigenvectors produces values above and below , so the point is a saddle point of a scalar function. These exhaust a nonsingular Hessian matrix.
A singular Hessian matrix does not always make the classification impossible: nonzero eigenvalues of both signs still give a saddle. If the Hessian matrix is positive or negative semidefinite, higher-order terms in its null directions must be examined. For example, and have the same Hessian matrix at zero, but the first has a strict minimum and the second a saddle. The function has a non-strict minimum along a whole line.
For the given polynomial,A stationary point has and then . Hence the origin is always critical, and two additional points exist exactly for :The Hessian matrix isAt the origin it is , giving a strict local minimum for and a saddle for . At either extra critical point it has determinant , so both extra points are saddles.
At , complete the square:Along it is positive away from zero, whereas along it is negative away from zero. Thus the origin remains a degenerate saddle despite its nonnegative Hessian matrix. This is the parabolic quartic critical-point classification. The complete result isFor the local minimum is not global, since along the same parabola .





