= Augmented Lagrangian method
{title2=$\lambda_i^+=\max(0,\lambda_i-g_i/\mu)$}
For smooth constraints $g_i\geq0$, minimize
$$
\Phi(x,\lambda,\mu)=f(x)+\frac1{2\mu}\sum_i
\bigl(\min(g_i(x)-\mu\lambda_i,0)^2-\mu^2\lambda_i^2\bigr)
$$
approximately in $x$, then update the nonnegative <Lagrange multipliers> by the displayed projected rule. Equality constraints instead contribute $-\nu_jh_j+h_j^2/(2\mu)$ and use the unprojected update $\nu_j^+=\nu_j-h_j/\mu$. The construction combines <Lagrangian duality> with a <quadratic penalty method>.
For an inequality, the exact identity is $\min(g_i,\mu\lambda_i)=\mu(\lambda_i-\lambda_i^+)$. Its unshifted negative part satisfies $\min(g_i,0)=\mu\min(\lambda_i-\lambda_i^+,0)$. On a violated constraint, accurate new multipliers therefore give violation of the order of $\mu$ times the multiplier error. This can achieve high feasibility accuracy without a vanishingly small $\mu$, reducing <condition number> difficulties. The negative-part qualification is essential if an iterate approaches a binding constraint from the feasible side.
Back to article page
