Exact penalty method

An exact penalty method uses a nonsquared residual such as $\Vert Au-f\Vert +\alpha J(u)$. Under a source condition it can recover an exact constrained minimizer for every sufficiently small fixed positive $\alpha$.

Suppose $A{u}^{†}=f$ and $p^{†}=A^{\ast }{\mu }^{†}\in \partial J(u^{†})$. For $F_{\alpha }(u)=\Vert Au-f\Vert +\alpha J(u)$, the subgradient inequality and dual pairing bound giveThus $0<\alpha \Vert {\mu }^{†}\Vert <1$ forces every minimizer to solve $Au=f$ exactly. Comparison of penalties then shows it is a J-minimizing solution, and $D_J^{p^{†}}(u,u^{†})=0$. The conclusion also holds for all $\alpha >0$ when ${\mu }^{†}=0$. One-homogeneity is not needed for this argument.