Newton's method for minimizing a twice differentiable function uses

A nonlinear programming problem minimizes an objective over finitely many real variables subject to constraints, with at least one objective or constraint nonlinear. Parametrizing the pulse amplitudes in quantum optimal control produces such a problem. Gradient descent and related local methods exploit differentiability, but a local stationary point need not be a global optimum without additional convexity or other structure.

A feasible point satisfies every optimization constraint of an optimization problem. For the budget constraint example, the feasible set is exactlyIt is the line segment from $(0,10)$ to $(10,0)$, a compact set and a convex set. The point $(2,8)$ is feasible, with product $16$; $(6,6)$ has a larger product $36$ but violates the budget constraint. Feasibility answers whether a decision is allowed; optimality asks whether any allowed decision is better. The extreme value theorem guarantees that this continuous objective function attains its maximum on this nonempty compact set.

Eliminating $y$ gives a one-variable objective function:By completing the square, $q(x)\le 25$, with equality exactly at $x=5$. Alternatively its derivative is $q^{\prime }(x)=10-2x$, which vanishes at $5$; the endpoint values are both zero. Completing the square proves global optimality directly, whereas the first-order optimality condition only identifies a candidate. It also quantifies the cost of imbalance: moving to $(5+\delta ,5-\delta )$ with $|\delta |\le 5$ loses exactly ${\delta }^2$ units of product. The arithmetic-geometric mean inequality gives another short certificate: $\sqrt{xy}\le (x+y)/2=5$, with equality exactly when $x=y$.

For a transportation problem with capacity inequalities, feasibility requires nonnegative shipments, row totals no greater than factory capacities and column totals equal to shop demands. Feasibility alone does not prove minimal cost: a matching transportation dual certificate with capacity inequalities provides an optimality proof. The feasible set can be empty, and a finite infimum need not be attained even when the set is nonempty.

Heuristic optimization seeks useful feasible solutions without necessarily providing a universal approximation ratio or a polynomial worst-case guarantee. Examples are local search, simulated annealing and tabu search. A claimed approximation algorithm needs a separate proved performance guarantee.

For positive constants $A,\alpha ,\beta$, a Cobb–Douglas production function describes output in terms of capital $K$ and labour $L$. It is homogeneous of degree $\alpha +\beta$. Under $K+wL=b$ with $b,w>0$, its unique nonnegative-input optimum is $K=\alpha b/(\alpha +\beta )$ and $L=\beta b/[w(\alpha +\beta )]$, obtained by maximizing the strictly concave logarithm along the budget line. The optimized output has derivative $(\alpha +\beta )\varphi (b)/b$, equal to its budget Lagrange multiplier. When $\alpha +\beta >1$, the unconstrained Lagrangian supremum over the positive orthant is infinite for every finite multiplier, despite the finite constrained optimum; the global dual representation must not be presumed in this case.

Maxima and minima are extrema of a real-valued function. A global maximum of $f$ on its domain $D$ is a point $x_{\ast }$ with $f(x)\le f(x_{\ast })$ for every $x\in D$; a global minimum reverses the inequality. A local extremum requires the inequality only in a neighbourhood of the point within $D$. In constrained optimization, $D$ is the feasible set, so an extremum is tested relative to the constraints rather than in every ambient direction. Thus a constrained local maximum or local minimum can occur at a boundary point without being an interior critical point.

Project scheduling asks how quickly a collection of dependent activities can finish. Adding their durations usually overestimates the answer, because independent activities can run together; taking only the longest single activity usually underestimates it, because some activities must wait. The critical path method identifies the chain of dependencies that determines the finish time. Its simplest model assumes known, fixed durations, unlimited parallel processing, and precedence requirements saying that each predecessor must finish before its successor starts. These are the assumptions of the basic model in Hendrickson, Project Management for Construction, §10.2.

Represent each activity by a vertex of a finite directed acyclic graph; an edge $i\to j$ means that activity $j$ waits for activity $i$. Activity $i$ runs without interruption for duration ${\tau }_i\ge 0$, starting at $t_i\ge 0$. The optimization constraints and project finish time areThe objective function is to minimize $T$. All durations and start times use the same time unit, and the project starts at time zero. The directed acyclic graph assumption allows a topological order in which every predecessor is processed first.

Consider this small, abstract project:

Activities A and E may start together. After A finishes, B and C may run together, but D must wait for both. F must similarly wait for both D and E: a merge means all incoming requirements must hold. This activity-as-vertex convention keeps durations on the activities and uses arrows only to express precedence.

Unlimited parallel processing is essential to the equality between the longest graph path duration and the optimal finish time. For example, two independent chains, each containing two activities of duration 3, have longest graph path duration 6. Two processors can finish both chains in 6, but one processor needs 12 because all four activities must run sequentially. Thus with limited resources the longest graph path still gives a lower bound, but it need not be achievable. Such resource requirements belong to the broader subject of scheduling.

// Target: mathematical-optimization.bigb

An objective function is a real-valued function assigning a score to each decision under consideration. An optimization problem seeks a feasible solution maximizing or minimizing this score. The optimizing decision and its score are different objects: in the budget constraint example, the decision is $(5,5)$ and the optimal value is $25$.

Choosing an objective function expresses what matters. With $x+y=10$ and $x,y\ge 0$, maximizing $x+y$ makes every feasible point equally good; maximizing $x$ selects $(10,0)$; maximizing $xy$ rewards balance. If $x,y$ are side lengths, $xy$ is a rectangle's area, and the budget constraint fixes its perimeter at $20$. The same mathematics then says that a square encloses the largest area among these rectangles.

For the product objective function, the positive level sets $xy=c$ are hyperbolas. Each collects decisions with identical scores. The highest attainable level set touches the feasible set at $(5,5)$; higher ones miss it entirely.

The product $xy$ is neither a convex function nor a concave function on the positive quadrant: along $(t,t)$ it is $t^2$, while along $(t,10-t)$ it is $10t-t^2$. Yet its restriction to this feasible set is a strictly concave function. Thus the geometry of the allowed decisions can make an otherwise nonconcave maximization problem tractable. Replacing $f$ by $-f$ always converts a maximization problem into an equivalent minimization problem.

// Target: mathematical-optimization.bigb

Scheduling assigns jobs to processing resources and orders their execution subject to resource and precedence requirements. Objectives include completion time, lateness and throughput. In single-machine sequencing with all jobs mandatory, the total processing time is constant, but order-dependent setups can still change completion time. With parallel machines, machine allocation and sequencing jointly determine the parallel-machine makespan with sequence-dependent setups.

Compartmental models divide a biological population into states and specify transfers between them. A compartment might contain susceptible individuals, infective individuals or recovered individuals. The model must state what a compartment represents and whether its variable is a count or a fraction.

For a closed, homogeneously mixed SIR model, let $N=S+I+R$ and use force of infection $\beta I/N$ and recovery rate $\gamma$. ThenEvery transfer leaves one compartment and enters another, so summing the equations gives $\dot{N}=0$. This conservation of population follows from the specified closed model; demographic gains, deaths or external arrivals can change the total. An SEIR model adds an exposed state to distinguish infection from the ability to transmit.

A deterministic ordinary differential equation describes continuous compartment sizes. A stochastic epidemic model instead treats infection and recovery as random events between integer states. The latter can exhibit epidemic extinction even when the corresponding deterministic trajectory initially grows.

Roots of unity among the linearization multipliers give periodic linearized perturbations but do not by themselves prove a branch of nonlinear periodic solutions. For the normalized delayed recurrence $F_r(x,y)=(y,r(1+y)/(1+(r-1)(1+x{)}^2)-1)$, the multipliers at $r=2$ are $e^{\pm i\pi /3}$. Nevertheless, with $Q=x^2-xy+y^2$, its sixth iterate has leading displacement $(r-2)Bv+QDv$, where $B=\left(\begin{array}{cc}1& 1\\ -1& 2\end{array}\right)$ and $D=\left(\begin{array}{cc}-1/2& -1\\ 1& -3/2\end{array}\right)$. The identity $\det (Bv,QDv)=Q^2/2$ obstructs cancellation for small nonzero $v$. Indeed any putative local six-cycle requires $r-2=O(|v{|}^2)$ by the norm equation, and then its determinant equation would read $0=Q^2/2+O(|v{|}^5)$, impossible. This is a concrete caution against promoting a linear resonance to a nonlinear bifurcation assertion.

Here $r>0$ is the low-density per-generation multiplication factor and $b>0$ sets the inverse crowding scale. The positive fixed point is $(\sqrt{r}-1)/b$ when $r>1$; its derivative $2/\sqrt{r}-1$ lies strictly between minus one and one, so it is locally asymptotically stable. The map reaches its maximum $r/(4b)$ at $N=1/b$ and decreases thereafter. Stability follows from the derivative, not from assuming that every overcompensating population map undergoes period doubling.

The Ricker population map models density-dependent reproduction with a declining per-capita growth factor. Its nonzero equilibrium is $K$, with multiplier $1-r$. It loses stability through a period-doubling bifurcation at $r=2$. Every population after the first update is bounded by $K{e}^{r-1}/r$, the maximum of the update function on nonnegative populations.

Population dynamics studies changes in the abundance and composition of a biological population through births, deaths, movement and interactions. The first step is a balance: count gains and losses over the same time interval and within the same population boundary. If each individual gives birth at constant per capita rate $b$ and dies at rate $d$, with no movement across that boundary, the deterministic abundance $N$ obeysThe coefficients have units of inverse time, so $b>d$ gives exponential growth rather than addition of a fixed number each generation.

Resources and crowding can introduce density dependence. For example, the logistic differential equation uses $\dot{N}=rN(1-N/K)$, with carrying capacity $K$. An age-structured population equation or stage-structured population model separates individuals whose survival and reproduction differ. At low abundance, a birth-death process also represents demographic stochasticity: a positive deterministic growth rate does not prevent a finite population from going extinct. Repeated counts and survival or reproductive observations test which rates and state variables the model actually needs.

A seed-producing population with first-season and second-season recruitment leads to nonnegative recurrence coefficients $a,b$. Its characteristic roots are $(a\pm \sqrt{a^2+4b})/2$. The nonnegative dominant root is below one precisely when $a+b<1$, giving deterministic decay of every finite initial population. This is a mean-population model; literal integer extinction requires a stochastic interpretation or a separate integer-population argument.

A chemostat maintains a well-mixed culture at constant volume $V$ by feeding fresh nutrient solution and removing culture at the same volumetric flow $Q$. Its dilution rate is $D=Q/V$. Mixing makes the outflow's nutrient concentration and biomass concentration equal to those inside the vessel; continuous replacement supplies nutrient and also removes growing cells.

The simplest one-nutrient chemostat uses nutrient concentration $s(t)$, biomass concentration $x(t)$, constant feed nutrient concentration $s_{\mathrm{in}}>0$, and constant biomass yield $Y>0$. Assume a cell-free feed, one growth-limiting nutrient, constant volume, and no cell retention, death or additional nutrient consumption for maintenance. With specific growth rate $\mu (s)$, its ordinary differential equations areThe nutrient balance contains inflow $D{s}_{\mathrm{in}}$, outflow $Ds$, and consumption $\mu (s)x/Y$; the biomass balance contains production $\mu (s)x$ and removal $Dx$. If $s$ is in grams of nutrient per litre and $x$ in grams of dry biomass per litre, $Y$ is grams of dry biomass produced per gram of nutrient consumed. Thus both balances have units of their respective concentration per time. The Monod equation supplies an empirical choice of $\mu (s)$; the balances themselves express the stated production, consumption and transport assumptions.

The appeal of a chemostat is that, at a positive chemostat steady state, adjusting the pump sets the specific growth rate to $D$. The residual nutrient concentration then adjusts to that rate, and the available nutrient together with biomass yield sets the biomass concentration. This gives a controlled way to compare cultures at selected growth rates and a simple quantitative warning of chemostat washout.

Two 1950 publications on continuous culture are Novick and Szilard's Description of the Chemostat, Science 112(2920), 715-716, dated 15 December, and Monod's La technique de culture continue. Théorie et applications, Annales de l'Institut Pasteur 79, 390-410. The latter's original bibliographic details are reproduced in the contents of Monod's collected papers. These references identify contemporary publications without establishing priority between them. ^[ref]. ^[ref].

The Monod equation is an empirical saturating law for the specific growth rate of a culture limited by one nutrient. Write $s\ge 0$ for its residual nutrient concentration, ${\mu }_{\max }>0$ for the limiting specific growth rate, and $K_s>0$ for the nutrient concentration at half that rate:Here $\mu$ and ${\mu }_{\max }$ have units of inverse time, whereas $K_s$ has the same units as $s$. The notation $K(C)$ in the title denotes the same growth function, with $C=s$, $K_{\max }={\mu }_{\max }$ and $K_0=K_s$. For $s\ll K_s$, $\mu (s)\approx ({\mu }_{\max }/K_s)s$; for $s\gg K_s$, $\mu (s)$ approaches ${\mu }_{\max }$. Its derivative ${\mu }^{\prime }(s)={\mu }_{\max }{K}_s/(K_s+s{)}^2$ is positive: more nutrient increases the specific growth rate, with progressively smaller increments.

For example, take ${\mu }_{\max }=1{\mathrm{h}}^{-1}$ and $K_s=1\mathrm{g}{\mathrm{L}}^{-1}$. Nutrient concentrations of $0$, $1$ and $9\mathrm{g}{\mathrm{L}}^{-1}$ give specific growth rates $0$, $0.5$ and $0.9{\mathrm{h}}^{-1}$. In a chemostat with dilution rate $D=0.5{\mathrm{h}}^{-1}$, the intersection $\mu (s)=D$ selects the residual nutrient concentration $s=1\mathrm{g}{\mathrm{L}}^{-1}$, provided the feed can supply enough nutrient for a positive biomass concentration.

The Monod equation is useful because two measurable parameters describe both nutrient scarcity and saturation. It assumes the other conditions affecting growth remain controlled and does not represent inhibition at high nutrient concentration. Although its formula resembles the Michaelis-Menten equation, a whole culture's specific growth rate combines many processes; the resemblance alone does not derive it from Michaelis-Menten kinetics or identify $K_s$ with a Michaelis constant.

In his 1949 review, Monod presents the hyperbolic rate law as an empirical fit in equation (2), pp. 383-384, and discusses the difficulty of identifying a single growth-controlling reaction. His reported division rates use doublings per unit time; the natural-exponential specific growth rate used here is that division rate multiplied by $\ln 2$. ^[ref].

A network describing how gene products affect the production of other gene products. Positive and negative regulatory interactions can generate multistability, oscillations, and noise-sensitive transitions.

// Target: mathematical-biology.bigb

A food web records resource-consumer interactions among species. A linear food chain is a special case. Positive interaction coefficients alone do not imply persistence: the per-capita growth balances must admit feasible positive long-term mean abundances.

An Allee effect makes per-capita growth less favorable at low abundance. A strong effect permits a stable zero state and a stable positive state separated by a finite threshold. Cooperative infections can exhibit the same positive-feedback structure: neither disease invades alone, but sufficiently many joint infections sustain both.