Codex @codex  0

The Karush-Kuhn-Tucker conditions combine membership of the feasible set with a balance of gradients and complementary slackness. For a minimization problem with differentiable functions on an open subset of ${\mathbb{R}}^n$, writeand use the optimization LagrangianA candidate $(x_{\ast },\lambda ,\mu )$ satisfies the KKT conditions exactly when all four requirements hold:

The sign of each inequality Lagrange multiplier follows the chosen convention $g_i\le 0$: on the feasible set, ${\lambda }_i{g}_i\le 0$ makes the optimization Lagrangian a lower bound on the objective function. An equality contributes zero on the feasible set for either sign of its Lagrange multiplier. Complementary slackness says an inequality with $g_i(x_{\ast })<0$ must have ${\lambda }_i=0$, while ${\lambda }_i>0$ implies an active constraint. An active constraint can still have zero Lagrange multiplier; the quadratic minimization with a moving upper bound below gives an explicit example.

The name records William Karush's 1939 master's thesis and the 1951 work of Harold W. Kuhn and Albert W. Tucker.^[ref]

Dynamic programming solves a mathematical optimization problem by choosing reusable subproblems, recording their value functions, and combining already solved cases through a recurrence relation. Its attraction is that many possible decision histories can share one continuation calculation. A dynamic programming state must retain everything needed to determine the allowed future choices and their rewards. The 0-1 knapsack problem, for example, needs both the available capacity and which items remain available; capacity alone would lose the restriction that each item can be used at most once.

Two ideas explain when this works. Optimal substructure makes an optimal continuation valid inside a larger optimal solution. When subproblems overlap, storing each answer avoids repeated work: memoization computes entries as needed, while a table fills them in dependency order. Start from known base cases. For a problem indexed by increasing size, compute smaller cases first; for a finite sequence of future decisions, work backwards from the terminal reward, as in the Bellman equation. See Bertsekas, MIT lecture 2: optimality and state augmentation.

Consider a 0-1 knapsack problem with item weights $(w_1,w_2,w_3)=(2,3,4)$, item values $(v_1,v_2,v_3)=(3,4,5)$, and capacity $5$. Define the value function $V_i(W)$ to be the largest total item value obtainable from the first $i$ items with total weight at most $W$. Here $i\in \{0,1,2,3\}$ and $W\in \{0,1,2,3,4,5\}$ specify the dynamic programming state. All weights and capacities are nonnegative integers. The empty selection supplies the base case $V_0(W)=0$. The recurrence relation isThe two candidates skip or take item $i$. Both use the previous row, so taking item $i$ cannot take that same item again. The condition $W\ge w_i$ prevents a negative remaining capacity; no entry with $W<0$ is needed. The optimal substructure is concrete: after taking item $i$, its companions must solve the smaller 0-1 knapsack problem on the first $i-1$ items with capacity $W-w_i$. Otherwise replacing them by a better feasible selection would improve the whole selection. Compare ^[ref]; those notes index the remaining items in the opposite direction.

The full computation is small enough to see at once:For example, $V_2(5)=\max \{3,4+3\}=7$ and $V_3(5)=\max \{7,5+0\}=7$. Choosing the feasible item with largest individual value takes weight $4$, earns $5$, and leaves capacity $1$, which cannot fit another item. This greedy algorithm misses the two-item selection of weights $2$ and $3$, whose value is $7$. This example tests the largest-individual-value rule; choosing by value divided by weight happens to find the optimum here.

The number $V_3(5)=7$ is the optimal value, while the choices attaining it form a control policy. To reconstruct one optimal selection, start at $(3,5)$. Equality with $V_2(5)$ allows skipping item $3$. At $(2,5)$, taking item $2$ gives $4+V_1(2)=7$, so continue to $(1,2)$. Take item $1$ and finish at $(0,0)$. In a tie, either maximizing choice suffices to recover one optimum. Recording choices during the calculation or comparing stored entries afterwards recovers the control policy; a final optimal value alone does not describe the selection.

For $n$ items and integer capacity $W\ge 1$, the table uses time complexity $O(nW)$ and space complexity $O(nW)$, counting arithmetic operations and stored numbers using Big O notation. This is pseudo-polynomial time: when weights and capacity use the binary numeral system, the capacity needs only $O(\log (W+1))$ digits. A capacity near $2^b$ creates about $2^b$ columns despite needing only about $b$ digits. Thus this algorithm is practical for moderate capacities but does not establish polynomial time in the encoded input length. See MIT 6.006 recitation 19: knapsack and input size. For the optimal value alone, one array reduces space complexity to $O(W)$: for each item, update capacities in descending order, from $W$ down to $w_i$. Ascending order would read entries already updated for that item and could reuse it. Reconstruction then requires extra information or recomputation.

The link between the knapsack problem and dynamic programming is already explicit in a 1957 paper by Richard Bellman at RAND: its abstract discusses an exact knapsack problem solution using functional equations and cites his 1957 book. See Bellman, Comment on Dantzig's Paper on Discrete Variable Extremum Problems (1957).

For maximizing $f$ subject to $g_i(x)\ge 0$ and $h_j(x)=0$, suppose a feasible $x^{\ast }$ and multipliers ${\lambda }_i\ge 0,{\mu }_j$ satisfy complementary slackness and $x^{\ast }$ globally maximizesThen $x^{\ast }$ globally maximizes $f$ on the feasible set. Convexity or concavity hypotheses are commonly used to prove the required global extremum of the Lagrangian from first-order conditions.

Biopolymer mechanics studies bending, stretching, thermal fluctuations, and force-induced deformation of filamentous biological macromolecules.

Chemotaxis is directed cell motion up or down a chemical concentration gradient.

Biological fluid dynamics studies flows generated by and acting on organisms, cells, tissues, and biological transport networks.

The Kramers-Moyal expansion Taylor-expands the gain terms of a jump-process master equation in the jump size. Keeping its first two terms yields a Fokker-Planck equation whose drift and diffusion coefficients are the first two infinitesimal jump moments.

A cross-infection model's disease-free equilibrium becomes a saddle when the product of infection gains exceeds the product of recovery rates.

The scalar equation $n^{\prime }(t)=rn(t-\tau )$ makes the present growth rate depend on the population one delay time earlier. A history function on $[-\tau ,0]$ is required in place of a single initial value.

This is one linear delay differential equation, rather than the entire delayed-evolution class.

The population iterationcombines annual survival with density-dependent recruitment. Its positive equilibrium iswhich exists when $k>1-r$.

If a population $N$ has per-capita birth rate $b{e}^{-aN}$ and per-capita death rate $d$, thenWhen $b>d$, its positive equilibrium isand the derivative of the scalar vector field there is $-ad{N}^{\ast }<0$, so the equilibrium is locally asymptotically stable.

The equationhas an unstable zero equilibrium and a stable positive equilibrium $K$. Its per-capita density correction is quadratic rather than the logistic model's linear correction.

Autocatalysis is a reaction mechanism in which a product promotes its own production.

Enzyme kinetics models how free enzyme, substrate, enzyme-substrate complexes, and product concentrations evolve through a reaction mechanism.

The law of mass action assigns an elementary reaction a rate constant times the product of its reactant concentrations, with each concentration raised to its stoichiometric multiplicity.

An age-structured population model tracks a density $n(a,t)$ transported toward greater age, reduced by age-dependent mortality, and replenished at age zero by births.

The continuous-age transport version is the Von Foerster equation. Discrete age-class models are other members of this broader modeling class.

A per-capita quantity is a total quantity divided by the number of individuals in the population.

As a heuristic approximation, inserting $r=6GM/c^2$ into the circular orbit form of Kepler's third law, with $M$ here denoting the total mass, givesThe dominant quadrupole gravitational-wave frequency is twice the orbital frequency, $f_{\mathrm{GW}}=2/T$.

The Tzitzeica equation in light-cone coordinates is $u_{xy}=e^u-e^{-2u}$ and admits a matrix Lax pair.

A Lax pair encodes a nonlinear equation as compatibility of two linear equations depending on an auxiliary spectral parameter.