On the strictly feasible set , use the logarithmic barrierChoose a strictly feasible starting point and parameters . For each parameter minimize the barrier functional by an unconstrained method confined to , warm-starting from the preceding point. In the exact version take a stationary point of the barrier subproblem. The stationarity equation isThese are the natural Lagrange multiplier estimates; the basic method need not perform a separate primal-dual correction.
The barrier method KKT convergence theorem is a global stationarity result: if the exact barrier stationary points form a bounded sequence, then every cluster point satisfying the linear independence constraint qualification is a KKT point. To prove it, take any subsequence . Continuity gives feasibility of . For an inactive index, and therefore .
The active gradients at are independent. The supplied dual-basis result gives bounded vectors , converging as , with for active . Pair stationarity with these vectors:Thus all active multipliers are bounded as well. Extract a convergent multiplier subsequence. Its limit is nonnegative, satisfies stationarity by continuity, and satisfies complementarity because . This proves the claimed KKT conclusion. The same proof permits approximate barrier stationary points whose stationarity residual tends to zero. Bounded iterates and the constraint qualification are substantive hypotheses, not automatic consequences of writing a barrier functional.
For a global-minimum conclusion one needs global minimization of the barrier subproblems and appropriate access to every feasible point. Specifically, suppose globally minimizes , has a cluster point , and . For any fixed strictly feasible ,Along the convergent subsequence the are bounded above, so taking the upper limit gives . Density of strictly feasible points and continuity of extend the inequality to every feasible . Hence is a global minimizer under these stronger hypotheses.
The constraint set is compact, so the continuous objective attains a maximum and minimum. The two constraint gradients are nonzero and supported on separate coordinates, so they are linearly independent everywhere on it. The Lagrange multiplier equations areBoth positive and negative values occur. At a maximizing or minimizing point the objective is therefore nonzero, so , , and both are nonzero. The equations imply , , and . The constraints then give , .
Consequently, writing , the complete list of extrema isFor an independent global check, Cauchy-Schwarz inequality gives , with equality exactly at the displayed points.
The rows of the Jacobian matrix are . Rank means precisely that these gradients are linearly independent. The Lagrange multiplier condition is that real numbers exist withGeometrically, is perpendicular to the tangent space of the constraint level set, whose tangent vectors form . The full-rank hypothesis ensures this level set is a smooth local constraint manifold.
There are real Lagrange multipliers such thatEquivalently . The rank assumption says the two constraint gradients are linearly independent; it rules out an abnormal constraint point.
Use constraints and . Both constraints and the objective are continuously differentiable. The feasible set is a nonempty closed subset of a bounded ellipsoid, hence compact, so the extreme value theorem guarantees both extrema.
The gradients and are independent everywhere feasible. If the first were times the second, then , , . The plane constraint would give , forcing the origin, which is not on the ellipsoid. Thus the Lagrange multiplier theorem applies at every feasible point.
Write the variables as , with . The hypothesis makes invertible. The implicit function theorem supplies neighbourhoods of and a unique smooth map such thatIts derivative is . Thus the derivative of this parametrization has image exactly .
Since has a constrained maximum at , differentiating gives zero in every direction. Therefore vanishes on . In finite-dimensional linear algebra, is the row space of . ConsequentlyThe full-rank hypothesis is what permits this Lagrange multiplier conclusion; it is not a consequence of constrained maximality alone at a singular constraint.
The Lagrange multiplier condition says that at a constrained local extremum of a continuously differentiable objective on a regular surface , its gradient is normal to the tangent plane: for some scalar multiplier.
Past exam of the options II course of the University of Oxford 2008 printed variant F F2 c Solution 2026-10-31
The Lagrange multiplier condition says that at a constrained local extremum of a continuously differentiable objective on a regular surface , its gradient is normal to the tangent plane: for some scalar multiplier.
For fixed endpoints and a regular integral constraint, Lagrange multipliers replace the constrained stationarity problem by stationarity of , with constant . Writing , the Euler-Lagrange equation is . Regularity here means the first variation of is not identically zero; abnormal constraint extrema otherwise need separate consideration.
If has no explicit dependence, thenThis is the Beltrami identity, giving the required constant of integration.
For the chain, , so its fixed arc length is exactly , and its gravitational energy is . The positive constant can be suppressed in minimization. UseThe equation, with a consistent sign convention for the multiplier, isThe first integral becomes . It implies ; solving givesThe equal endpoint heights force , by evenness and strict increase of on the positive half-line. Thus the extrema have the asserted catenary form. The length and height conditions giveFor the hanging minimum choose , the upward-convex branch lying below its endpoints. For this positive is unique: writing , the ratio increases continuously from one to infinity. The negative- branch describes an upward arch, not the hanging minimum. If , only the horizontal straight chain is admissible and is the limiting degenerate case; is impossible.
A embedded submanifold of dimension is locally the graph of a function of coordinates after a change of coordinates. Equivalently, near each point it is a regular level set of a scalar function with .
The Lagrange multiplier theorem says that if a function has a constrained local extremum at on such a level set, thenfor some real . Indeed the derivative of must vanish on the tangent space , whose normal space is the line spanned by . The nonzero-gradient condition is essential.
Work on the positive orthant, where all real powers in are defined. To justify a global maximum, enlarge the constraint set to the compact simplexIt is closed and bounded since . The product extends continuously to , is zero at every boundary point where a coordinate vanishes, and equals at . Hence a maximum exists and lies in the positive interior. For the constraint gives directly.
For , at an interior maximum the Lagrange multiplier equations areAll and are positive, so all are equal. The constraint and make their common value . Thus the only possible maximizer has value , provingwith equality exactly when every . The source writes on all of ; for general noninteger exponents its relevant real-valued domain is the positive orthant used here.
At a regular constrained extremal, the first variations satisfy for a constant Lagrange multiplier . One way to justify this is to choose a variation for which , use it to correct any other variation to preserve the constraint by the implicit function theorem, and differentiate along that corrected family. This gives the same ratio for all variations. The exceptional case for every variation requires the usual separate abnormal-extremal analysis.
Set . Integration by parts in its first variation givesTaking compactly supported variations and applying the fundamental lemma of the calculus of variations yields the Euler-Lagrange equation. For the quadratic integrands, it isor with . If endpoint values are free, independent endpoint variations give the natural conditionsAt a prescribed endpoint the variation instead vanishes and no natural condition is added there.
Under these natural conditions, integration by parts givesHence . For two solutions satisfying the same natural conditions, subtract their cross-multiplied equations and integrate:The final equality is the vanishing endpoint boundary form. Thus distinct eigenvalues giveThe same argument works for other common homogeneous boundary conditions making that boundary form zero.
In the Legendre case, . Comparing coefficients of in a degree- solution gives , so different degrees have different eigenvalues. A degree-zero solution is . A linear solution must be orthogonal to the nonzero constant solution, so and . Write . Orthogonality to gives , and orthogonality to gives . ThereforeThe constants are free normalization factors; for solutions of the stated exact degrees they are nonzero. The natural endpoint conditions hold automatically for these polynomials because .
For a regular isoperimetric constraint, introduce a constant Lagrange multiplier and set . With fixed endpoint values, the Euler-Lagrange equation isIf and have no explicit dependence, the Beltrami identity gives the first integralThe regularity assumption excludes the exceptional case in which the constraint itself has zero first variation for every admissible perturbation.
Parametrize the surface of revolution by . The magnitude of is . Integrating the resulting surface area element over , and integrating the circular cross-sectional areas for the volume, givesHere because it is a radius. For a regular connected closed surface, take in the interior; the closing endpoints can have unbounded even when the surface itself is smooth.
After dividing by , its integrand is . The Euler-Lagrange equation and Beltrami identity becomeAs tends to zero at an endpoint, also tends to zero, so . On the positive interior this implies and, writing ,The Euler-Lagrange equation also yields . Thereforeincluding at the maximum where . Integrating and using the first integral givesThus the regular extremal is a sphere. The two zeros fix its center and radius:Equivalently, the prescribed volume fixes . If are fixed in advance, their separation and the prescribed volume must be compatible with these equations; there is no positive regular stationary profile of the stated type for an incompatible volume.
For minimality, the three-dimensional isoperimetric inequality states that every sufficiently regular closed surface enclosing volume haswith equality precisely for a sphere. Our extremal has and , so it attains equality and is a global minimum of area at the fixed volume. This also makes clear the connected-surface assumption behind uniqueness: allowing several disconnected components introduces stationary collections of spheres, rather than the single positive generating profile used here.
Ignoring a factor independent of the parameters, the likelihood function isSince every count is positive, a boundary value has zero likelihood. At an interior optimum, a Lagrange multiplier for gives . Summing the resulting gives , soThe log likelihood function is strictly concave on the positive simplex, because its Hessian matrix is diagonal with entries . Hence this critical point is the unique global maximum-likelihood estimate.
Under independence, the log likelihood function separates:Maximizing the two terms over their respective probability simplices, by the same Lagrange multiplier argument as in part (a), givesTherefore the constrained maximum-likelihood estimates areThe associated fitted cell counts are .
Take a smooth variation with on the boundary. Differentiating under the integral gives the first variationIntegration by parts removes the derivatives of , and the boundary contribution vanishes. The fundamental lemma of the calculus of variations therefore givesFor a regular constraint , the Lagrange multiplier rule says that an extremal satisfies for a constant . Equivalently, apply the preceding Euler–Lagrange equation to . Regularity means that is not identically zero as a linear functional on the admissible variations. For the quadratic constraint below this holds at every nonzero admissible ; a zero constraint leaves only .
In polar coordinates the constrained integrand isIts Euler–Lagrange equation is , orThe field is single-valued and periodic in . Substitution with givesThe centre is part of the disk, so one must also select the regular radial solution: for , smoothness gives and the regular derivative behaviour; for , is bounded and . A singular solution at zero does not define an admissible smooth field on the disk.
For fixed , let , so and for . The angular derivative term has integral , including zero when . ConsequentlyThe one-dimensional Euler–Lagrange equation for isexactly the same radial equation. The common factor cancels.
For two regular solutions with the same and different multipliers, multiply the equation for by , multiply that for by , subtract and integrate. The result isAt , both functions vanish; at zero, regularity makes the boundary term tend to zero. Since the multipliers are distinct,This is weighted Sturm–Liouville orthogonality with weight .
Finally, multiply one radial equation by and integrate by parts. The same endpoint conditions yieldMultiplication by provesFor any nonzero admissible solution the energy on the left before subtraction is positive, so its multiplier is the positive Rayleigh quotient .
On the constraint set, , so . Part (a)(iv) and the Lagrange multiplier theorem give , henceSince is nonzero, it is an eigenvector. Moreover , so the associated eigenvalue is real and nonnegative.
The Lagrange multiplier theorem for one regular constraint says: if are near , , , and is a local extremum of restricted to , thenFor several constraints with linearly independent gradients, the objective gradient lies in their linear span. The regularity assumption rules out singular points where the usual multiplier conclusion can fail.
Ignoring a factor independent of the probabilities, the likelihood function is , so its log is subject to . A Lagrange multiplier gives . Summing makes .
ThusSince all counts are positive, a zero fitted cell probability would have zero likelihood. On the interior the log-likelihood is strictly concave, so this stationary point is its unique global maximum.
On the constraint set, , so . Part (a)(iv) and the Lagrange multiplier theorem give , henceSince is nonzero, it is an eigenvector. Moreover , so the associated eigenvalue is real and nonnegative.