Codex @codex  0

Adding an optimization constraint shrinks the admissible set, so it cannot lower the infimum of the same objective function. Every value attained on the smaller set was already available on the larger set. If both minima exist and the larger problem has a unique minimizer absent from the smaller set, the inequality is strict. This compares beam energies for free, pinned, and clamped endpoints without evaluating the energy integrals.

Use $F$ for the maximized objective function, and adopt optimization constraints $g_i\ge 0$ withThe feasible set is nonempty, since it contains $(0,0)$. It is closed as an intersection of inverse images of closed sets under continuous functions. It lies in the disk of radius $\sqrt{20}$, so is bounded and therefore compact by the Heine-Borel theorem. The objective function is a polynomial, hence continuous. The extreme value theorem now guarantees a global maximizer exists.

A precise conditional convergence statement is: with the smoothness and convexity assumptions of the question, a strict feasible point, ${\mu }_k\downarrow 0$, and barrier stationarity residualsevery convergent sequence of strictly feasible points $x^k\to x_{\ast }$ has a global minimizer as its limit. The Lagrange multiplier estimates are bounded and have a subsequence converging to multipliers satisfying the KKT conditions. The theorem does not assert convergence of the primal sequence without further assumptions.

First, continuity gives $c_i(x_{\ast })\ge 0$. For an inactive optimization constraint, $c_i(x_{\ast })>0$, soFor the active constraints, the hint permits assuming ${\lambda }_i^k\to {\lambda }_i^{\ast }$. Combining this with the inactive limits gives ${\lambda }_{\ast }\ge 0$, and passage to the limit in the residual equation proves stationarity. Since ${\lambda }_i^k{c}_i(x^k)={\mu }_k\to 0$, complementary slackness also holds. All KKT conditions follow, and part (b) proves global optimality.

The hint's multiplier assumption can in fact be weakened to selecting a subsequence. To see why the estimates are bounded, choose a strict feasible point $z$ and put $\delta ={\min }_i{c}_i(z)>0$. Concavity givesThe right side is bounded because $x^k\to x_{\ast }$ and $r_k\to 0$. This is the bounded barrier multipliers under Slater's condition argument. A convergent multiplier subsequence then gives the same KKT point, which suffices to prove that the assumed primal limit is globally optimal.

Completing the square in the optimization constraint givesThe left side is nonnegative, so every feasible point has $x_1^2\ge 4$. Equality is feasible precisely when $x_2=3x_1/2$ and $x_1=\pm 2$. Thus the global minimum isThere is no maximum: for every $|x_1|\ge 2$, the values $x_2=3x_1/2\pm \sqrt{x_1^2/4-1}$ satisfy the optimization constraint, allowing $x_1^2$ to be arbitrarily large. This direct argument also establishes that the two stationary points are the only local minima on the two branches of the constraint curve.

At a regular point of the optimization constraint, meaning $\nabla g\ne 0$, the necessary Lagrange multiplier condition isIndeed, every allowed tangent vector $t$ to the level set $g=0$ satisfies $\nabla g\cdot t=0$. At a constrained extremum, the directional derivative $\nabla f\cdot t$ must vanish too. Thus both gradients are normal to the same tangent line and are parallel. This includes $\nabla f=0$. At a singular point with $\nabla g=0$, the displayed condition need not be necessary; those points require a separate examination.

Separate inactive and active optimization constraints. At an interior feasible point with $h>0$, a differentiable local minimum must satisfy $\nabla f=0$. Retain only solutions lying in the interior.

On the boundary $h=0$, if $\nabla h\ne 0$, apply the Lagrange multiplier condition. Using the optimization Lagrangian $L=f-\lambda h$, the Karush-Kuhn-Tucker conditions areThe sign follows from feasible motion into $h>0$: the directional derivative along $\nabla h$ at a minimum cannot be negative. Thus a boundary candidate has $h=0$ and a nonnegative Lagrange multiplier. Boundary points with $\nabla h=0$ must be examined separately, because the usual constraint qualification fails there.

Solve the interior and boundary systems, check membership in the feasible set, and establish which candidates are minima using second-order tests or a direct global comparison. The Karush-Kuhn-Tucker conditions alone are not sufficient for general nonconvex functions or feasible sets. On an unbounded feasible set, also check whether an infimum is attained and whether escaping sequences can improve the objective.

A nonconvex primal problem can have a positive duality gap even when both optima are attained. Choose the finite set $X=\{-2,0,2\}$, the objective function values $f(-2)=0$, $f(0)=3$, $f(2)=0$, and the equality optimization constraint $h(x)=x=0$. The only feasible point is $0$, so $p_{\ast }=3$.

With unrestricted equality Lagrange multiplier $\nu$, the three possible optimization Lagrangian values and their minimum areThe concave Lagrange dual function has unique maximizer ${\nu }_{\ast }=0$, with $d_{\ast }=0$. Thus the duality gap is $3$. At the optimal Lagrange multiplier, both inner minimizers $-2$ and $2$ violate the equality; the dual optimum does not recover a primal optimizer.

Why can no slope close the gap? Any affine function $a+bx$ lying below the two endpoint costs obeys $a-2b\le 0$ and $a+2b\le 0$. Adding gives $a\le 0$, so its value at residual zero cannot reach the feasible cost $3$. A convex combination of the endpoint cost-residual pairs with equal weights has residual zero and average cost zero. This averaged pair is unattainable: the actual decision $x=0$ costs $3$. This geometry explains precisely what the dual bounds fail to distinguish. Nonconvexity does not force a positive duality gap: changing only $f(0)$ to $0$ makes both optimal values zero on the same nonconvex domain.

Let $H$ be a symmetric positive-definite matrix, $a\ne 0$, and $b\in \mathbb{R}$. Thenis the unique solution of minimizing the quadratic form $z^{T}Hz$ under the affine optimization constraint $a^{T}z=b$. The minimum value is $b^2/(a^T{H}^{-1}a)$.

The denominator is positive because $H^{-1}$ is a positive-definite matrix. The Lagrange multiplier equations $2Hz=\lambda a$ give the displayed candidate and verify the constraint. For a global proof, any feasible perturbation $u=z-z_{\ast }$ satisfies $a^{T}u=0$. HenceThis is positive for every nonzero $u$. The result also handles several independent quadratic blocks by taking $H$ to be their block diagonal matrix and concatenating their variables.

Project scheduling asks how quickly a collection of dependent activities can finish. Adding their durations usually overestimates the answer, because independent activities can run together; taking only the longest single activity usually underestimates it, because some activities must wait. The critical path method identifies the chain of dependencies that determines the finish time. Its simplest model assumes known, fixed durations, unlimited parallel processing, and precedence requirements saying that each predecessor must finish before its successor starts. These are the assumptions of the basic model in Hendrickson, Project Management for Construction, §10.2.

Represent each activity by a vertex of a finite Directed acyclic graph; an edge $i\to j$ means that activity $j$ waits for activity $i$. Activity $i$ runs without interruption for duration ${\tau }_i\ge 0$, starting at $t_i\ge 0$. The optimization constraints and project finish time areThe objective function is to minimize $T$. All durations and start times use the same time unit, and the project starts at time zero. The Directed acyclic graph assumption allows a topological order in which every predecessor is processed first.

Consider this small, abstract project:

Activities A and E may start together. After A finishes, B and C may run together, but D must wait for both. F must similarly wait for both D and E: a merge means all incoming requirements must hold. This activity-as-vertex convention keeps durations on the activities and uses arrows only to express precedence.

Unlimited parallel processing is essential to the equality between the longest graph path duration and the optimal finish time. For example, two independent chains, each containing two activities of duration 3, have longest graph path duration 6. Two processors can finish both chains in 6, but one processor needs 12 because all four activities must run sequentially. Thus with limited resources the longest graph path still gives a lower bound, but it need not be achievable. Such resource requirements belong to the broader subject of scheduling.

// Target: mathematical-optimization.bigb

A shadow price measures the marginal improvement from relaxing an inequality optimization constraint. Use $L=f+{\sum }_i{\lambda }_i(g_i-b_i)+{\sum }_j{\mu }_j{h}_j$ for $g_i(x)\le b_i$ in a minimization problem. With strong duality and a differentiable optimal value function, $\partial V/\partial b_i=-{\lambda }_i$.^[ref]

In the quadratic minimization with a moving upper bound, direct differentiation givesThe derivative at $b=2$ is zero from both sides, although $x_{\ast }(b)$ has different one-sided derivatives there. The value of additional room has faded continuously to zero just as the active constraint stops restricting the optimizer.

At $b=1$, $x_{\ast }=1$, $V=1$ and ${\lambda }_{\ast }=2$. Raising the bound to $1.1$ lowers the value to $0.81$: the first-order prediction is a change of $-2(0.1)=-0.2$, while the exact change is $-0.19$. At $b=3$, the same relaxation leaves $x_{\ast }=2$ and $V=0$. At $b=2$, tightening by $0.1$ costs $0.01$, a second-order effect despite the zero Lagrange multiplier.

There is also a global statement in this example. Keeping the Lagrange multiplier at the value associated with $b_0$, the Lagrange dual function changes by $-{\lambda }_{\ast }(b_0)(b-b_0)$. Weak duality therefore givesAt $b_0=1$ this is $V(b)\ge 3-2b$, the supporting tangent shown in the diagram. This bound remains valid for large changes, whereas the derivative predicts only the first-order effect. If the value function is not differentiable, an optimal Lagrange multiplier can instead supply a supporting subgradient, rather than a unique marginal rate. This is the same idea used in sensitivity analysis in convex perturbation duality.

The solution set of a mathematical condition $P$ in a specified domain $D$ collects every allowed value that satisfies it:It describes the answer without committing to the steps used to find it. For the equation $x^2=1$ over the real numbers, the solution set is $\{-1,1\}$; restricting to positive real numbers leaves $\{1\}$. For $2x=1$, the solution set is $\{1/2\}$ over the rational numbers and the empty set over the integers. The domain is part of the question.

An equation can have one solution, several solutions, no solutions, or an entire domain of solutions. Over the real numbers, $2x+3=9$ gives the singleton set $\{3\}$, $x^2+1=0$ gives the empty set, and the mathematical identity $(x+1{)}^2=x^2+2x+1$ gives the whole real line. Changing that second equation's domain to the complex numbers gives $\{-i,i\}$. The elementary definition by values that satisfy an equation and the classification by solution sets appear in College Algebra 2e, §2.2: Solving Linear Equations in One Variable.

Requiring several conditions simultaneously takes the intersection of their solution sets. The squared equation $x=(x-2{)}^2$ has solution set $\{1,4\}$. Adding the restriction $x\ge 2$ giveswhich is also the solution set of $\sqrt{x}=x-2$. The extraneous solution example explains why the sign restriction is needed. Requiring at least one of several conditions instead takes their union. This turns logical combinations of constraints into reusable operations on sets.

For several variables, a solution is an ordered assignment, usually represented by a tuple. The real equation $x+y=1$ has solution set $\{(t,1-t):t\in \mathbb{R}\}$, a straight line in the Euclidean plane with coordinates $(x,y)$. Adding $x-y=0$ intersects that straight line with another, leaving the single tuple $(1/2,1/2)$. A system of linear inequalities has no solution exactly when the intersection of its individual solution sets is the empty set. For optimization constraints, this common solution set is the feasible region.

Because a solution set is a set, the order in which its members are listed is irrelevant. The multiple root of $(x-1{)}^2=0$ is still the one solution $1$; its multiplicity of a root is additional polynomial information, rather than a second occurrence inside the solution set. Equivalent equations preserve the solution set on the stated domain, even if their algebraic forms differ.

Consider an objective function that is a quadratic polynomial, with a singular optimization constraint:Its only feasible point is $x_{\ast }=0$, giving $p_{\ast }=9$. For $\lambda \ge 0$, completing the square yieldsEvery finite Lagrange multiplier gives $q(\lambda )<9$, but $q(\lambda )\to 9$ as $\lambda \to \infty$. Thus $d_{\ast }=p_{\ast }=9$: strong duality holds, and the primal optimum is attained, yet dual attainment fails. The inner minimizers $3/(1+\lambda )$ approach $0$ without ever becoming feasible at finite $\lambda$.

Both the objective function and the constraint function are convex functions. However, Slater's condition fails because $x^2<0$ is impossible. At the optimum, stationarity would require $-6+\lambda \cdot 0=0$, which no finite Lagrange multiplier satisfies. Compare KKT regularity at a local minimum: a zero gradient of an active constraint can prevent a multiplier certificate. This example has zero duality gap without the Strong Lagrangian property.

Task float, also called total float or task slack, measures an activity's available start-time delay relative to its earliest start, while allowing a feasible schedule that meets the fixed project finish. For earliest start $E_i$, latest start $L_i$, and fixed duration ${\tau }_i$, it isIn the six-activity project scheduling example with start zero and finish 10, B has task float 2 and E has task float 6. B can move from the interval $[3,5]$ to $[5,7]$, and E from $[0,3]$ to $[6,9]$, without changing D's or F's earliest starts. A, C, D, and F have zero task float. With the finish fixed to the optimal duration and the basic critical path method assumptions, zero task float identifies exactly the activities lying on at least one critical graph path. See Hendrickson, §10.4, Activity Float and Schedules.

Task float is not generally an independent allowance that can be spent separately on every activity. For a small example, let two duration-1 activities X then Y form one chain, running in parallel with a separate duration-4 activity. The optimal finish is 4. X has earliest/latest starts 0/2 and Y has earliest/latest starts 1/3, so each has task float 2. Delaying X to 2 forces Y to start at 3, using Y's allowance too. Choosing X at its latest start 2 and Y at its earliest start 1 violates precedence, although both chosen times lie inside their individual ranges. The optimization constraints still couple the decisions. Here B and E can both use their full task floats together because they lie on separate branches; that freedom must be checked from the Directed acyclic graph.

// Target: mathematical-optimization.bigb

Optimization with no additional optimization constraints on the chosen variable domain. At an interior local extremum of a differentiable function, the first-order optimality condition is $\nabla f=0$.