Optimization Lagrangian ($L(u,\lambda ,\mu )$)

For a minimization problem with $g_i(u)\le 0$ and $h_j(u)=0$, the optimization Lagrangian is $L=f+{\sum }_i{\lambda }_i{g}_i+{\sum }_j{\mu }_j{h}_j$ with ${\lambda }_i\ge 0$ and unrestricted equality Lagrange multipliers. Its infimum over the original variable domain gives a dual lower bound. For a maximization problem, use $L=f-{\sum }_i{\lambda }_i{g}_i-{\sum }_j{\mu }_j{h}_j$ to obtain upper bounds. The Lagrangian sufficiency theorem combines a global extremum of this function with feasibility and complementary slackness. This is distinct from a Lagrangian density in variational physics.

For equality-constrained optimization, restrict the Hessian matrix $H$ of the optimization Lagrangian to feasible tangent directions. If the columns of $Z$ form a basis of the nullspace of the constraint Jacobian matrix $A$, the resulting quadratic form has matrix $Z^{T}HZ$. Positive definiteness gives positive quadratic curvature in every nonzero tangent direction. Nonsingularity of this reduced matrix, together with full row rank of $A$, characterizes nonsingularity of the corresponding equality-constrained saddle-point matrix.