Penalty method

A penalty method incorporates constraint violation into the objective with a parameter controlling its cost. For an equality constraint $c(x)=0$, the quadratic penalty method adds a multiple of $\Vert c(x){\Vert }_2^2$; a power penalty uses higher even powers. The penalty can allow infeasible iterates while encouraging feasibility as the cost grows. This differs from a logarithmic barrier, whose domain itself excludes the constraint boundary and exterior. Existence of a penalized stationary point, convergence of its constraint violation and convergence of the primal points are separate questions.

For an equality-constrained optimization problem, an even integer $p\ge 2$ makes this a nonnegative penalty for constraint violation. Its derivative is $\nabla f+{\mu }^{-1}D{c}^T{c}^{p-1}$, with the power applied coordinatewise. The case $p=2$ is the quadratic penalty. Reducing $\mu$ increases the cost of infeasibility; it does not itself ensure that computed stationary points or minimizers stay bounded.

At a stationary point of the power penalty, define the displayed multiplier estimates. Its stationarity equation becomes $\nabla f-D{c}^{T}y=0$. If $\mu \downarrow 0$ and these multipliers are bounded, then each $c_i\to 0$. Along any convergent primal subsequence, a further multiplier subsequence converges and yields equality-constrained first-order KKT conditions at the limit. This is a statement about finite cluster points: bounded multipliers alone do not prove convergence or boundedness of the primal iterates.