ABBA measurement sequence 2026-10-18
Alternating sample and reference readings can suppress instrument drift. In this original four-reading model, measure objects at equally spaced times . Assume the objects are stable during that short sequence and each reading is its object's length plus the same affine function of time, . Then
because both pairs have mean time . The constant offset and linear drift cancel. With a further drift term , direct substitution instead leaves bias . A shorter sequence reduces that term, but neither this ordering nor averaging corrects unequal temperatures, sample changes or reference measurement uncertainty. This is a modern illustrative protocol; the historical account below establishes alternation, without attributing this particular four-reading sequence to Charles-Édouard Guillaume.
Affine expression 2026-10-20
An expression of the form , with fixed coefficients . It contains a constant term and terms of degree one, without products of variables. It represents an affine function once the scalar domain is specified.
Let and be the universal covering maps of two one-dimensional complex tori. Every holomorphic map has a holomorphic lift between one-dimensional complex tori satisfying , and every such lift is an affine function
with . Indeed, for , the continuous function takes values in the discrete set and is therefore constant. Hence is -periodic. It is bounded on a compact fundamental parallelogram and therefore on the complex plane, so the Liouville theorem makes constant. If preserves the identity and the lift is chosen with , then and the lift is a linear map.
Affine linear classifier 2026-10-31
An affine linear classifier assigns binary labels according to the sign of an affine function. Its boundary is an affine hyperplane, and its positive region is a closed half-space under the displayed convention. On the full class has VC dimension : arbitrary signs interpolate on affinely independent points, while the intersecting convex hull partition theorem prevents shattering any points. Requiring gives the homogeneous case and a different VC dimension.
Affine minorant Created 2026-10-06 Updated 2026-11-05
An affine minorant of a function on is an affine function satisfying everywhere. Every proper convex function that is lower semicontinuous on has an affine minorant: separate a point strictly below its closed epigraph by a hyperplane. The finiteness of somewhere makes the separating vertical coefficient strictly positive, so the separator can be solved for height.
This supplies a useful existence estimate. For and ,
The proximal objective is therefore coercive and lower semicontinuous; in finite dimensions it attains a minimum. Its strong convexity makes that minimizer unique, giving a well-defined proximal operator.[ref] The finite-dimensional attainment argument should not be applied to arbitrary infinite-dimensional normed spaces merely from boundedness of a minimizing sequence.
Affine-weighted hat mass matrix Created 2026-10-06 Updated 2026-11-05
On a one-dimensional uniform finite element mesh of spacing , let be an affine function and the interior piecewise-linear hat functions. The weighted mass matrix has entries
and zero entries for nonadjacent indices. The diagonal product is symmetric about and integrates to ; the neighbouring product is symmetric about and integrates to . The odd part of an affine function integrates to zero around each centre, giving the formulas. This evaluates a variable reaction term exactly without substituting a constant coefficient or a quadrature approximation.
Barycentric coordinate system Created 2026-10-18 Updated 2026-10-31
A barycentric coordinate system assigns to a point in the affine span of the vertices of a nondegenerate simplex its unique vertex weights , constrained to sum to one. Uniqueness follows because form a basis of the translation space: expand in that basis and set . A point lies in the closed simplex exactly when every weight is nonnegative. An affine function satisfies , which makes these coordinates useful for interpolation and the integral of an affine function over a simplex.
Bernstein polynomial Created 2026-09-24 Updated 2026-10-14
For a function and an integer , its Bernstein polynomial is
The Bernstein basis supplies nonnegative weights whose sum is one. Equivalently, the weights are the probabilities of a binomial distribution, and the result is an expected value. At a given , this averages the sampled values with weights concentrated near .[ref]
The endpoints are exact: and . Interior sampled points need not be interpolated. For example, binomial distribution moments give and . Thus every affine function is reproduced exactly, whereas is generally changed. Bernstein polynomial degree preservation is a statement about degree of a polynomial, not equality of the entire polynomial.
For a continuous , the uniform continuity of and the variance explain uniform convergence. Split the expected value of the error into , where uniform continuity makes the change small, and its complement, whose probability is at most by Chebyshev's inequality. The bounded function bounds the remaining error. This proves convergence without assuming differentiability.
For , the tensor-product Bernstein polynomial is the expected value of for independent random variables . The same uniform continuity and coordinatewise variance estimates give uniform convergence for each fixed finite .
Conditional Jensen inequality Created 2026-10-05 Updated 2026-11-05
Let be a real integrable random variable, let be a sub-sigma-algebra, and let be a finite convex function with integrable. Then
almost surely. These explicit integrability assumptions make both conditional expectations well defined.
A finite convex function on the real line is the supremum of a countable family of supporting affine functions, for example choosing supports at rational points and using continuity. Write them as . For each , monotonicity and linearity of conditional expectation give . Countability permits one exceptional null set for every . Taking the supremum proves the inequality.
With and square-integrable , the inequality says that retaining only the information in cannot increase its second moment. More generally it is the convex-loss step behind the Rao-Blackwell theorem.
In Cartesian coordinates on four-dimensional Minkowski spacetime, all Conformal Killing vector fields have the form
Their conformal Killing factor is . There are four translation generators, six Lorentz transformation generators, one dilation generator, and four special conformal transformation generators. These form a fifteen-dimensional Lie algebra under the Lie bracket of vector fields.
For completeness, differentiate the Conformal Killing equation in three index orders to obtain
Its divergence gives ; its tensor contraction gives . Hence is an affine function. Subtracting the displayed quadratic and dilation terms leaves a Killing vector field whose second partial derivatives vanish, proving completeness of the list. The vector fields are globally smooth, but some special conformal transformations have incomplete flows on affine Minkowski spacetime; the dilation flow is global.
Convex conjugate Created 2026-09-28 Updated 2026-10-14
The convex conjugate records a function using slopes rather than input positions. Let be a proper extended-real function: it is finite somewhere and never takes . Its convex conjugate is
where is the dot product and is the effective domain. An infinite value of excludes that input from the supremum. The value can be , but cannot be because an input with finite supplies a finite lower bound.
The convex conjugate is automatically a convex function, even when is not convex: each fixed input gives an affine function , and their supremum is convex. It is also lower semicontinuous. It need not be proper: for on , is unbounded above for every . The definition and the automatic convexity are discussed in Boyd and Vandenberghe, Convex Optimization (2004), §3.3.1.
The best lower bound with a prescribed slope. Whenever is finite, the affine function
is an affine minorant of . Indeed, an affine function lies below everywhere exactly when . Thus is the largest possible intercept at slope . The convex conjugate trades positions for slopes: specifies where to inspect , whereas specifies which global affine minorant to use. Taking the supremum over all these affine minorants produces the biconjugate; the Fenchel-Moreau theorem says this recovers a proper convex function when it is lower semicontinuous.
A quadratic, computed by completing the square. For the convex quadratic function ,
Consequently , and the unique maximizing input is . The affine minorant is the tangent line at : its slope is and its intercept is . For example, gives and the tangent line , touching at . This is the same curve described either by its point values or by the intercepts of its tangent lines.
Figure 1.
The convex conjugate determines the intercept of each tangent line of a convex quadratic function
. For the convex quadratic function , the colored tangent lines have slopes and touch at . The right panel plots the convex conjugate and marks the matching slope-value pairs. The highlighted tangent line has intercept . Details.
A corner turns into a constraint on slopes. For the absolute value , if then , with value zero at . If , taking makes ; if , taking does the same. Therefore its convex conjugate is zero on and outside: it is the indicator functional of a constraint set for this interval. For the unique maximizing input is zero; for every maximizes, and for every maximizes. The corner's many supporting slopes are exactly the subdifferential .
A supremum need not be attained. For a differentiable convex function finite on all of , an input maximizes exactly when . If is strictly convex, there is at most one such input; existence is a separate issue. For the exponential function , the derivative is positive. At , solving gives and . At , the supremum of is zero as , with no maximizing input. At , as . Thus a strictly convex differentiable function can have both unattained finite conjugate values and infinite conjugate values.
Convex function Created 2026-09-24 Updated 2026-10-14
A function on a convex set is a convex function if, for every and ,
Geometrically, a line segment joining two points of its graph of a function lies on or above that graph of a function. The convex set condition ensures that the intermediate input stays in the domain of a function. An affine function satisfies the defining inequality with equality.
For the quadratic function , the inputs and have equally weighted arithmetic mean . Evaluating at the arithmetic mean gives , whereas averaging the outputs gives . Averaging first saves one unit in this example. More generally, direct expansion gives
This identity proves convexity and also shows exactly when its defining inequality is strict.
Figure 1.
The graph of a convex quadratic lies below its endpoint interpolation and above its supporting tangent
. For , the line segment through and has height at , while the graph of a function has height . The tangent line is an affine minorant: everywhere. Details.
For a differentiable convex function, the tangent line in one dimension, or the corresponding supporting affine function in several dimensions, lies below the entire graph of a function. This is the subgradient inequality. The equivalent geometric statement is that the epigraph, consisting of points on or above the graph of a function, is a convex set. These characterizations are developed in Boyd and Vandenberghe's convex-function slides, sections 3.1–3.4.
Convexity does not require differentiability: the absolute value function is a convex function with a corner at . When a function is twice continuously differentiable on an open interval, it is a convex function exactly when its second derivative is nonnegative throughout that interval. In several dimensions, the corresponding criterion on an open convex set is that its Hessian matrix is positive semidefinite everywhere. The domain of a function matters: is a convex function on , but not on all of . A strictly positive second derivative is sufficient for strict convexity, but is not necessary: is a strictly convex function although its second derivative vanishes at .
The usefulness of convexity is that local information can certify a global conclusion. Every local minimum of a convex function on a convex set is a global minimum: if a lower value existed elsewhere, the defining inequality would produce lower values arbitrarily close to the proposed local minimum. For a differentiable convex function on an open convex set, a zero gradient therefore certifies a global minimum. Strict convexity gives uniqueness when a global minimum exists; it does not supply existence by itself. These facts explain the role of convex functions in convex optimization.
For differentiable functions that are convex functions on an open convex set and affine functions , the KKT conditions certify a global minimum. Once Lagrange multipliers satisfy them, no extra Slater condition is needed.[ref]
The certificate is a short chain of inequalities. Nonnegative Lagrange multipliers and affine equalities make a convex function. Its zero gradient gives a global minimum at by the first-order optimality condition. For every feasible point ,
The first equality uses feasibility and complementary slackness; the last inequality uses and . Equivalently, the Lagrange dual function attains the same value as the primal objective function, proving strong duality. A strictly convex function makes the primal optimum unique, although the Lagrange multipliers need not be unique.
Affine equality optimization constraints matter: an arbitrary signed multiple of a nonlinear equality can destroy convexity of the optimization Lagrangian. For example, minimizing subject to the affine equality gives and for . This equality Lagrange multiplier is positive for , zero at , and negative for . Completing the square gives the certificate for every real number .
Outside convex optimization, KKT stationarity alone does not certify an optimum: minimizing with no optimization constraints gives the stationary candidate , which is a global maximum. The Lagrangian sufficiency theorem remains applicable whenever a global minimum of the optimization Lagrangian can be proved directly.
Convex optimization Created 2026-09-24 Updated 2026-10-13
Convex optimization is a minimization problem with a convex function as its objective function and a convex set of feasible points. For a finite-valued , the problem is to find with for every . The displayed infimum describes its value even when no minimizing point exists. The two convexity requirements concern different objects: the allowed decisions and the scores assigned to them.
A convex set contains the whole line segment between any two of its points:
A convex function lies below the interpolation of its endpoint values:
Thus mixing two allowed decisions preserves feasibility, and the mixed score is no worse than the same mixture of endpoint scores. The objective function may have flat regions or corners. For a strictly convex function, the second inequality is strict when and .
A useful constraint representation is
where are convex functions on a common convex set , and the equalities are affine functions. Each inequality defines a convex sublevel set; their intersection with and the equality set is a convex set. An equality involving a general convex function need not preserve this geometry: allows and but excludes their midpoint . See Boyd and Vandenberghe, Convex Optimization (2004), §4.2.1 for the standard problem form.
Why local information can certify a global answer. Every local minimum of a convex optimization problem is a global minimum, including minima on the boundary. If a feasible point had , points would remain feasible and have value strictly below for every . Taking arbitrarily small contradicts a local minimum at . This argument explains the special power of convexity: a better distant decision would reveal an improving direction immediately nearby.
If a minimizer exists and the objective function is strictly convex on , it is unique. Two distinct minimizers of value would have a feasible midpoint with value strictly below . Existence is a separate question: the exponential function is strictly convex on , but its infimum is approached only as . Even a bounded convex set need not suffice: on also has unattained infimum . A continuous function on a nonempty compact set attains its minimum by the extreme value theorem.
Nor does strict convexity imply differentiability. For , the squared term gives the strict inequality while the triangle inequality gives the weak inequality for the absolute value. At zero the left and right derivatives are and , so no derivative exists there, although zero is the unique minimizer. Such objectives require subgradients or proximal operators rather than an ordinary gradient at every point.
For a differentiable convex function on an open convex set containing , the constrained first-order condition is
Indeed, the subgradient inequality makes this sufficient; taking the one-sided derivative along feasible line segments makes it necessary. At an interior minimizer, both directions are feasible and the gradient must vanish. At a boundary minimizer it can be nonzero: it only needs to make every feasible direction nondecreasing. See Boyd and Vandenberghe, §§3.1.3 and 4.2.3.
A fully computed nearest-point problem. Project onto the half-space by minimizing half the squared Euclidean distance:
The Hessian matrix is the identity matrix, so the objective function is strictly convex. The unconstrained minimizer violates the inequality. With the convention , the optimization Lagrangian is , where the Lagrange multiplier satisfies . Its Karush-Kuhn-Tucker conditions are
The last two equations give , . The choice is infeasible, so complementary slackness forces . Consequently , giving , and . The Euclidean distance is , rather than the objective function value . The gradient at the answer balances ; the displacement from to is a normal vector to the boundary.
Figure 1.
The nearest feasible point is where an objective contour first touches the half-space boundary
. Details.
There is an exact certificate independent of numerical search. Completing squares gives
For every feasible point, implies . Thus every feasible objective is at least , and attains this bound. More generally the Lagrangian dual problem has objective for . Its maximum is also , illustrating strong duality through matching computed bounds. This makes the example a model for Euclidean projection onto a convex set and the projection step in projected gradient descent.
What the multiplier conditions require. For differentiable convex functions on an open convex set and equalities defined by affine functions, the KKT conditions combine feasibility, , complementary slackness , and stationarity
with unrestricted equality Lagrange multipliers . They suffice to certify a global minimum without an additional regularity assumption: the optimization Lagrangian is convex, stationarity minimizes it, and feasibility with complementary slackness gives the same certificate as above. To guarantee their necessity at an attained finite optimum, a regularity hypothesis suffices: Slater's condition requires some with and . The projection example has the strictly feasible point . See Boyd and Vandenberghe, §§5.2.3 and 5.5.3. This hypothesis guarantees suitable multipliers, not existence of a primal minimizer.
Without regularity, necessity can fail even for convex functions: minimize subject to . The sole feasible point is optimal, but stationarity would require . No Lagrange multiplier works. There is no strictly feasible point, and the gradient of the constraint vanishes at the answer.
The nonconvex contrast. The polynomial has derivative and second derivative . Numerically solving the cubic gives stationary points , , and . Their objective values are respectively , , and , with second derivatives , , and . Hence both outer points are local minima, and the middle point is a local maximum. Since at both ends, its global minimum exists and must be one of the stationary minima: it is the left one. A method stopping at the right local minimum misses a better value by approximately .
Figure 2.
A strictly convex quadratic has one minimum; a tilted quartic has two local minima with different values
. Details.
The practical appeal of convex optimization is the ability to check an answer globally using geometry and matching Lagrangian duality bounds. The projection certificate certifies every feasible alternative at once; the tilted quartic shows why a zero gradient alone cannot do that for general nonlinear optimization.
Historically, an early landmark is Kuhn and Tucker's [ref]. Its §3 explicitly states a regularity hypothesis for inequality-constrained Lagrange multipliers; its discussion of saddle points explains how concave functions support global conclusions for maximization. This documents an early formulation of the multiplier approach; it does not establish sole discovery of today's KKT conditions or of convex optimization as a whole.
Suppose is a differentiable convex function with coordinate Lipschitz continuity constant . Put , the Bregman divergence from the supporting affine function at . The subgradient inequality gives , and its coordinate Lipschitz constant is still . The coordinate descent lemma with step size yields
Since , rearrangement proves the bound. Continuous differentiability suffices; a Hessian matrix is unnecessary.
Target: gradient-descent.bigb, under Lipschitz gradient.
The displayed estimate, for all and real , defines a coordinate Lipschitz constant for the partial derivative in coordinate . It implies the coordinate descent lemma
by the fundamental theorem of calculus and integration of the Lipschitz continuity estimate. Subtracting an affine function leaves these constants unchanged.
The apparently weaker estimate along the singleton negative gradient direction is equivalent when every real step size is allowed: if the initial partial derivative is nonzero, any displacement can be written ; if it is zero but the final partial derivative is nonzero, reverse the endpoints; if both vanish the estimate is trivial.
Target: gradient-descent.bigb, under Lipschitz gradient.
An equality-constrained convex optimization problem has the form
where is convex and the constraint is affine. Its stationarity and feasibility equations can be combined into a monotone primal-dual operator.
Finite convex function as supremum of affine minorants Created 2026-10-14 Updated 2026-10-20
For a finite real convex function on an entire real vector space, apply the convex domination form of the Hahn-Banach theorem to and the zero functional on . It supplies a linear functional dominated by this translated function. The affine function lies below everywhere and equals it at . Taking the supremum proves the formula. These are algebraic affine minorants; continuity requires additional topological hypotheses.
Finite element 2026-10-05
A finite element specifies a reference region, a finite-dimensional space of local trial functions, and degrees of freedom that uniquely determine each such function. Assembling compatible copies over a finite element mesh gives a global trial space. For a one-dimensional linear element, the region is an interval, the functions are affine functions, and the degrees of freedom are the endpoint values.
On a shape-regular mesh in dimensions at most three, nodal piecewise-linear interpolation of satisfies
where is the largest element diameter and is independent of . For homogeneous Dirichlet boundary conditions, belongs to the conforming finite element space. Scaling each element to a reference element gives the estimate; shape regularity bounds the scaling constants, while subtraction of an affine function leaves an error controlled by the second derivatives.