Constrained optimization (source code)

= Constrained optimization
{wiki}

<Constrained optimization> minimizes or maximizes an <objective function> over a <feasible set> defined by equality or inequality constraints. In $\min f(x)$ subject to $g_i(x)\leq0$ and $h_j(x)=0$, a feasible optimum must balance improvement of the objective against the permitted directions of motion.

For example, minimizing $x^2+y^2$ subject to $x+y=1$ gives $x=y=1/2$, since $x^2+y^2=((x+y)^2+(x-y)^2)/2\geq1/2$. The unrestricted minimizer $(0,0)$ is infeasible. <Lagrange multipliers> and the <Karush-Kuhn-Tucker conditions> generalize this balancing argument; necessity requires appropriate hypotheses, and sufficiency needs extra structure such as <convexity>.<Convex Optimization by Stephen Boyd and Lieven Vandenberghe>[Chapters 4–5, optimization problems and optimality conditions]{ref}