Consider the minimization problemThe objective function is a strictly convex function with its unconstrained optimization minimum at . If , it decreases throughout the feasible set until the bound is reached. If , the feasible set already contains . HenceWith , the optimization Lagrangian and KKT conditions areSubstituting gives the unique inequality Lagrange multiplierFor , the active constraint has . For , there is strict slack and . At , the active constraint has zero Lagrange multiplier: the boundary happens to pass through the unconstrained optimum. Feasibility and complementary slackness must both be checked; solving only would allow infeasible candidates.
The same quadratic objective with active, just-touching and inactive upper bounds
. Details. An independent certificate follows by completing the square:The Lagrange dual function is a concave function of . Its maximum over occurs at and equals . Thus every case has an explicit lower bound attained by a feasible point; the KKT conditions provide a certificate that can be checked without searching all real numbers.
An integral constraint need not admit a normal Lagrange multiplier unless its first variation is nonzero. For the quadratic normalization in the title, the variation in direction has derivative , so the constraint is always regular at an admissible function. With free endpoint values this justifies the Euler-Lagrange equation for and its natural endpoint conditions. For , multiplication of the differential equation by gives .
Scalar multiplier certificate for a quadratic equality constraint Created 2026-10-07 Updated 2026-10-14
For and , minimize subject to . Adding a positive Lagrange multiplier times the constraint gives a positive-definite quadratic optimization Lagrangian, whose minimizer is and . Feasibility is the displayed scalar equation. Its left side strictly decreases from positive infinity to negative infinity on , so it has a unique root for every real . The Lagrangian sufficiency theorem certifies a unique global optimum on the possibly nonconvex equality surface.
An optimal dual variable bounds the effect of perturbing a convex value function:If is finite convex near zero, . A singleton subdifferential gives differentiability and a first-order expansion. For an upper-bound constraint relaxed by , the derivative equals the negative of the nonnegative Lagrange multiplier; relaxing the bound can decrease the minimum value.
Shadow price of an inequality constraint 2026-10-14
A shadow price measures the marginal improvement from relaxing an inequality optimization constraint. Use for in a minimization problem. With strong duality and a differentiable optimal value function, .[ref]
Convex Optimization by Stephen Boyd and Lieven Vandenberghe. §5.6.3, printed pp. 251–252, equation (5.58): local sensitivity
In the quadratic minimization with a moving upper bound, direct differentiation givesThe derivative at is zero from both sides, although has different one-sided derivatives there. The value of additional room has faded continuously to zero just as the active constraint stops restricting the optimizer.
At , , and . Raising the bound to lowers the value to : the first-order prediction is a change of , while the exact change is . At , the same relaxation leaves and . At , tightening by costs , a second-order effect despite the zero Lagrange multiplier.
The derivative of the optimal value is the negative inequality multiplier
. Details. There is also a global statement in this example. Keeping the Lagrange multiplier at the value associated with , the Lagrange dual function changes by . Weak duality therefore givesAt this is , the supporting tangent shown in the diagram. This bound remains valid for large changes, whereas the derivative predicts only the first-order effect. If the value function is not differentiable, an optimal Lagrange multiplier can instead supply a supporting subgradient, rather than a unique marginal rate. This is the same idea used in sensitivity analysis in convex perturbation duality.
For , write and . Integration by parts gives , while . The Cauchy-Schwarz inequality therefore gives the displayed sharp bound, with equality exactly when . A prescribed positive budget consequently givesThis proves the unique global optimum, rather than only stationarity from a Lagrange multiplier.
For a real symmetric matrix and nonzero real vector , the quotient rule and the quadratic form gradient give the displayed gradient. It vanishes exactly when , so the stationary points are eigenvectors. Equivalently, the Lagrange multiplier condition for optimizing on the unit sphere is . The smallest and largest eigenvalues give the global extrema; other eigenvectors can be stationary without being extrema, as a direction mixing larger and smaller eigenvalues changes the quotient in opposite ways.
Impose zero Dirichlet boundary conditions on an interval. The isoperimetric constraint is regular because . The Lagrange multiplier rule applied to gives the Sturm-Liouville equationMultiplying by and using integration by parts gives . Thus the multiplier is the stationary value of the original functional, or its Rayleigh quotient. This calculation does not require to have one sign; positive-weight spectral conclusions require additional hypotheses.
If two stationary functions have different multipliers, multiplying their equations crosswise and subtracting gives . The endpoint terms vanish. This proves the weighted orthogonality of Sturm-Liouville eigenfunctions. Higher stationary values need not be local minima of the constrained functional.
Strong duality without dual attainment 2026-10-18
Consider an objective function that is a quadratic polynomial, with a singular optimization constraint:Its only feasible point is , giving . For , completing the square yieldsEvery finite Lagrange multiplier gives , but as . Thus : strong duality holds, and the primal optimum is attained, yet dual attainment fails. The inner minimizers approach without ever becoming feasible at finite .
Both the objective function and the constraint function are convex functions. However, Slater's condition fails because is impossible. At the optimum, stationarity would require , which no finite Lagrange multiplier satisfies. Compare KKT regularity at a local minimum: a zero gradient of an active constraint can prevent a multiplier certificate. This example has zero duality gap without the Strong Lagrangian property.
Strong Lagrangian property Created 2026-10-14 Updated 2026-10-18
For a finite constrained optimum value and optimization Lagrangian , the strong Lagrangian property means that some finite Lagrange multiplier attains the dual lower bound exactly: . It includes dual attainment, but need not include primal attainment. It is equivalent to a non-vertical supporting hyperplane of a value function: a supporting slope gives , hence ; weak duality gives the opposite inequality. Conversely the infimum identity bounds every below by ; taking the infimum over gives the supporting inequality. No convexity assumption is needed for this equivalence.
Time-dependent holonomic constraint 2026-11-05
A time-dependent holonomic constraint is a positional restriction whose permitted configurations change explicitly with time. Here denotes the generalized coordinates. A bead confined to a moving wire is an immediate example: its allowed position is specified at each instant, but the wire carries the permitted set of positions through space. The distinction between actual motion and a virtual displacement at fixed time is essential.[ref]
Differentiating one smooth holonomic constraint along an actual trajectory givesA permitted virtual displacement, however, compares neighboring allowed positions at the same instant:The missing time term is not an approximation. Actual velocity and a tangent virtual displacement obey different conditions when the constraint moves. In the following power calculation, take to be fixed Cartesian coordinates, so explicit time dependence belongs to the constraint rather than to a moving coordinate system.
For an ideal constraint, its generalized forces may be written , with a Lagrange multiplier . They do zero virtual work, yet their actual constraint power isThus an ideal, frictionless moving constraint can exchange energy with the constrained body. Zero virtual work does not imply zero actual work. The actuator maintaining the imposed motion is part of the physical energy accounting.
For a worked apparatus model, put a bead of mass on a smooth horizontal wire whose height is prescribed as in a uniform downward gravitational field of magnitude . The wire permits horizontal motion and supplies a vertical normal force . The constraint is , and Newton's second law givesThe bead's mechanical energy, including gravitational potential energy, isbecause horizontal motion is unforced. For , and with , the wire exerts ; at it delivers to the bead. A vertical normal force is perpendicular to every allowed horizontal virtual displacement, but parallel to part of the actual velocity. Measuring wire motion and bead load would expose precisely the energy transfer that is lost by treating the moving wire as a stationary, workless support.
A tolerance-budget residual bound for multiplier bisection controls the physical bias left unused by a numerical allocation, rather than only the numerical error in its Lagrange multiplier. Use from monotonicity of the multiplier equation for capped tolerances. Suppose a positive bracket satisfies . Choosing the upper endpoint gives a feasible allocation, with residualChoosing the lower endpoint can violate the budget. The direction of monotonicity therefore matters even when both endpoints are numerically close.
For an a priori bound, put . On the entire bracket, each summand obeys Lipschitz continuity with constant at most , including across its breakpoint. HenceAfter bisections of an original bracket with , the corresponding worst-case bound is . This is conservative; evaluating the two endpoint budgets can give a much tighter residual certificate.
For an original stopping example, use , and in common normalized bias units. The bracket straddles the root. Its feasible upper endpoint leaves unused. One bisection evaluates and replaces the upper endpoint, reducing that unused amount to about . A residual allowance of is now met, despite a breakpoint lying inside the bracket.
This gives a thresholded stopping of a capped-tolerance multiplier search in the same units as the permitted bias. Floating-point comparisons also need an allowance for rounding if they are to serve as hard certificates. The underlying bias coefficients and cost model remain assumptions of cost-weighted allocation of baseline-response tolerances; numerical precision cannot validate missing physical terms. NIST's pulse-radiometer account motivates the offset/leakage constraints rather than prescribing this numerical stopping rule.[ref]
Calibration Service for Low-level Pulsed-Laser Radiometers at 1.06 µm: Pulse Energy and Peak Power by Rodney W. Leonhardt. §3.5, printed p. 8; §7.2, Leakage Effect
Universal kriging Created 2026-10-05 Updated 2026-10-18
Universal kriging allows an unknown regression drift:The design matrix has full column rank, its entries and are known, and the regression coefficients are unknown. For example, rows and target row allow a linear spatial trend. Requiring to have zero bias for every gives . With known positive-definite matrix observation covariance matrix , the minimizing weights solvewhere is a vector of Lagrange multipliers. Equivalently, the generalized least squares estimate and prediction areThe first term is the estimated trend alone; the second predicts the correlated regression residuals. With , the error variance isThe last, nonnegative term accounts for estimating the specified drift coefficients. Merely inserting a fitted drift into simple kriging and keeping its known-mean error variance would omit this term. For a Gaussian random field, the same expressions arise by integrating over drift coefficients in a diffuse prior distribution limit; this is an additional inferential interpretation, not an assumption required by the linear optimization.[ref]
Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams. §2.7, pp. 27–28, equations (2.38)–(2.42)
William Karush Created 2026-10-14 Updated 2026-11-05
William Karush obtained necessary conditions for inequality-constrained optimization in his1939 master's thesis. The result was independently rediscovered and published by Harold W. Kuhn and Albert W. Tucker, producing the name Karush-Kuhn-Tucker conditions.[ref]
https://www.math.u-szeged.hu/~hajnal/courses/MSc_Optimalizalas/2014/Cottle%20-%20William%20Karush%20and%20the%20KKT%20theorem.pdf. Richard W. Cottle, William Karush and the KKT Theorem, Documenta Mathematica extra volume2012, printed pp255–257
Their distinguishing idea is that an inequality constraint can be active or inactive: a nonnegative Lagrange multiplier pairs with its constraint through complementary slackness. A constraint qualification is needed for the usual necessary-condition theorem; a stationary Lagrangian equation alone is insufficient. Worldline einbein Created 2026-10-05 Updated 2026-10-06
An einbein specifies a one-dimensional metric tensor along a worldline. In the relativistic particle phase-space action it is the Lagrange multiplier for the mass-shell condition and transforms as under the corresponding canonical gauge transformation. After eliminating the momentum, its geometric interpretation depends on the normalization of the particle action.

