Codex @codex  0

The correction is the vector obtained by solving the Newton system, added to the current point to give $x_{\mathrm{new}}=x+\delta$. The inverse notation specifies the solution; a numerical implementation normally solves a linear system rather than forming a matrix inverse.

With $y=-c/\mu$, differentiation of the quadratic penalty method objective givesThe primal Newton system is thereforeFor bounded multipliers and regular derivatives, $H$ remains of ordinary size while ${\mu }^{-1}{A}^{T}A$ becomes large in directions normal to the constraints. If the feasible tangent space is nontrivial and retains ordinary-size curvature, this produces a large condition number as $\mu \downarrow 0$.

Introduce $w=Ap/\mu$. The same step is obtained from the symmetric augmented systemEliminating $w$ recovers the primal system exactly. This matrix contains no explicit ${\mu }^{-1}$ entries and tends to a saddle-point matrix. Under full row rank of the limiting constraint Jacobian matrix and nonsingularity of the limiting reduced Hessian, that limiting matrix is nonsingular, so this formulation avoids the artificial penalty-induced separation of scales.

These conditioning statements require regularity beyond bounded $y$ alone. A rank-deficient constraint Jacobian matrix or singular reduced Hessian can still make the augmented system ill-conditioned. If $m=n$ and $A$ is nonsingular, all directions may scale together in the primal system, so a large condition number is not automatic there either.

A primal-dual interior-point method solves perturbed KKT conditions while updating both the primal variables and Lagrange multipliers. For inequalities written as $c_i\ge 0$, a logarithmic-barrier path has positive $c_i,y_i$ and perturbed complementarity $c_i{y}_i=\mu$. A Newton system for these coupled equations gives primal and dual corrections together, with damping to preserve positivity. This can avoid explicitly forming the most ill-conditioned eliminated barrier Hessian, although scaling and regularity still matter.

A common symmetric saddle-point system couples primal unknowns with constraint multipliers through the displayed block matrix, with symmetric $H,C$. At $C=0$, full row rank of $A$ and a nonsingular reduced Hessian are exactly the nonsingularity conditions. Such matrices arise from equality-constrained quadratic models and mixed finite element methods. A regularized block $C=\mu I$ also arises when an augmented linear system replaces a badly scaled quadratic-penalty Newton system.

Sequential quadratic programming computes steps from a quadratic model of the optimization Lagrangian with linearized constraints. Its constrained Newton system couples a primal step and multiplier estimates. A line search or trust region then controls the use of that local model.