On the strictly feasible set , use the logarithmic barrier
Choose a strictly feasible starting point and parameters . For each parameter minimize the barrier functional by an unconstrained method confined to , warm-starting from the preceding point. In the exact version take a stationary point of the barrier subproblem. The stationarity equation is
These are the natural Lagrange multiplier estimates; the basic method need not perform a separate primal-dual correction.
The barrier method KKT convergence theorem is a global stationarity result: if the exact barrier stationary points form a bounded sequence, then every cluster point satisfying the linear independence constraint qualification is a KKT point. To prove it, take any subsequence . Continuity gives feasibility of . For an inactive index, and therefore .
The active gradients at are independent. The supplied dual-basis result gives bounded vectors , converging as , with for active . Pair stationarity with these vectors:
Thus all active multipliers are bounded as well. Extract a convergent multiplier subsequence. Its limit is nonnegative, satisfies stationarity by continuity, and satisfies complementarity because . This proves the claimed KKT conclusion. The same proof permits approximate barrier stationary points whose stationarity residual tends to zero. Bounded iterates and the constraint qualification are substantive hypotheses, not automatic consequences of writing a barrier functional.
For a global-minimum conclusion one needs global minimization of the barrier subproblems and appropriate access to every feasible point. Specifically, suppose globally minimizes , has a cluster point , and . For any fixed strictly feasible ,
Along the convergent subsequence the are bounded above, so taking the upper limit gives . Density of strictly feasible points and continuity of extend the inequality to every feasible . Hence is a global minimizer under these stronger hypotheses.
The constraint set is compact, so the continuous objective attains a maximum and minimum. The two constraint gradients are nonzero and supported on separate coordinates, so they are linearly independent everywhere on it. The Lagrange multiplier equations are
Both positive and negative values occur. At a maximizing or minimizing point the objective is therefore nonzero, so , , and both are nonzero. The equations imply , , and . The constraints then give , .
Consequently, writing , the complete list of extrema is
For an independent global check, Cauchy-Schwarz inequality gives , with equality exactly at the displayed points.
The rows of the Jacobian matrix are . Rank means precisely that these gradients are linearly independent. The Lagrange multiplier condition is that real numbers exist with
Geometrically, is perpendicular to the tangent space of the constraint level set, whose tangent vectors form . The full-rank hypothesis ensures this level set is a smooth local constraint manifold.
There are real Lagrange multipliers such that
Equivalently . The rank assumption says the two constraint gradients are linearly independent; it rules out an abnormal constraint point.
Use constraints and . Both constraints and the objective are continuously differentiable. The feasible set is a nonempty closed subset of a bounded ellipsoid, hence compact, so the extreme value theorem guarantees both extrema.
The gradients and are independent everywhere feasible. If the first were times the second, then , , . The plane constraint would give , forcing the origin, which is not on the ellipsoid. Thus the Lagrange multiplier theorem applies at every feasible point.
Its equations are
Here , since otherwise the first two equations require . Subtract twice the first equation from the second, and use and ; the result is , so . Then and the ellipsoid gives . The only candidates are therefore , with matching signs. Consequently
Write the variables as , with . The hypothesis makes invertible. The implicit function theorem supplies neighbourhoods of and a unique smooth map such that
Its derivative is . Thus the derivative of this parametrization has image exactly .
Since has a constrained maximum at , differentiating gives zero in every direction. Therefore vanishes on . In finite-dimensional linear algebra, is the row space of . Consequently
The full-rank hypothesis is what permits this Lagrange multiplier conclusion; it is not a consequence of constrained maximality alone at a singular constraint.
The Lagrange multiplier condition says that at a constrained local extremum of a continuously differentiable objective on a regular surface , its gradient is normal to the tangent plane: for some scalar multiplier.
First the box must be centred at zero. If the two face coordinates in one direction are , subtracting the ellipsoid equations for two vertices differing only in that coordinate gives . A nondegenerate box has , hence . Its positive half-sides therefore satisfy and its volume is .
The feasible closed octant is compact, the boundary gives zero volume, and some interior box has positive volume, so the maximum is interior. Apply multipliers to there:
Thus the three squared scaled coordinates are equal and sum to one. The unique positive solution gives
The Lagrange multiplier condition says that at a constrained local extremum of a continuously differentiable objective on a regular surface , its gradient is normal to the tangent plane: for some scalar multiplier.
First the box must be centred at zero. If the two face coordinates in one direction are , subtracting the ellipsoid equations for two vertices differing only in that coordinate gives . A nondegenerate box has , hence . Its positive half-sides therefore satisfy and its volume is .
The feasible closed octant is compact, the boundary gives zero volume, and some interior box has positive volume, so the maximum is interior. Apply multipliers to there:
Thus the three squared scaled coordinates are equal and sum to one. The unique positive solution gives
For fixed endpoints and a regular integral constraint, Lagrange multipliers replace the constrained stationarity problem by stationarity of , with constant . Writing , the Euler-Lagrange equation is . Regularity here means the first variation of is not identically zero; abnormal constraint extrema otherwise need separate consideration.
If has no explicit dependence, then
This is the Beltrami identity, giving the required constant of integration.
For the chain, , so its fixed arc length is exactly , and its gravitational energy is . The positive constant can be suppressed in minimization. Use
The equation, with a consistent sign convention for the multiplier, is
The first integral becomes . It implies ; solving gives
The equal endpoint heights force , by evenness and strict increase of on the positive half-line. Thus the extrema have the asserted catenary form. The length and height conditions give
For the hanging minimum choose , the upward-convex branch lying below its endpoints. For this positive is unique: writing , the ratio increases continuously from one to infinity. The negative- branch describes an upward arch, not the hanging minimum. If , only the horizontal straight chain is admissible and is the limiting degenerate case; is impossible.
A embedded submanifold of dimension is locally the graph of a function of coordinates after a change of coordinates. Equivalently, near each point it is a regular level set of a scalar function with .
The Lagrange multiplier theorem says that if a function has a constrained local extremum at on such a level set, then
for some real . Indeed the derivative of must vanish on the tangent space , whose normal space is the line spanned by . The nonzero-gradient condition is essential.
Work on the positive orthant, where all real powers in are defined. To justify a global maximum, enlarge the constraint set to the compact simplex
It is closed and bounded since . The product extends continuously to , is zero at every boundary point where a coordinate vanishes, and equals at . Hence a maximum exists and lies in the positive interior. For the constraint gives directly.
For , at an interior maximum the Lagrange multiplier equations are
All and are positive, so all are equal. The constraint and make their common value . Thus the only possible maximizer has value , proving
with equality exactly when every . The source writes on all of ; for general noninteger exponents its relevant real-valued domain is the positive orthant used here.
At a regular constrained extremal, the first variations satisfy for a constant Lagrange multiplier . One way to justify this is to choose a variation for which , use it to correct any other variation to preserve the constraint by the implicit function theorem, and differentiate along that corrected family. This gives the same ratio for all variations. The exceptional case for every variation requires the usual separate abnormal-extremal analysis.
Set . Integration by parts in its first variation gives
Taking compactly supported variations and applying the fundamental lemma of the calculus of variations yields the Euler-Lagrange equation. For the quadratic integrands, it is
or with . If endpoint values are free, independent endpoint variations give the natural conditions
At a prescribed endpoint the variation instead vanishes and no natural condition is added there.
Under these natural conditions, integration by parts gives
Hence . For two solutions satisfying the same natural conditions, subtract their cross-multiplied equations and integrate:
The final equality is the vanishing endpoint boundary form. Thus distinct eigenvalues give
The same argument works for other common homogeneous boundary conditions making that boundary form zero.
In the Legendre case, . Comparing coefficients of in a degree- solution gives , so different degrees have different eigenvalues. A degree-zero solution is . A linear solution must be orthogonal to the nonzero constant solution, so and . Write . Orthogonality to gives , and orthogonality to gives . Therefore
The constants are free normalization factors; for solutions of the stated exact degrees they are nonzero. The natural endpoint conditions hold automatically for these polynomials because .
For a regular isoperimetric constraint, introduce a constant Lagrange multiplier and set . With fixed endpoint values, the Euler-Lagrange equation is
If and have no explicit dependence, the Beltrami identity gives the first integral
The regularity assumption excludes the exceptional case in which the constraint itself has zero first variation for every admissible perturbation.
Parametrize the surface of revolution by . The magnitude of is . Integrating the resulting surface area element over , and integrating the circular cross-sectional areas for the volume, gives
Here because it is a radius. For a regular connected closed surface, take in the interior; the closing endpoints can have unbounded even when the surface itself is smooth.
After dividing by , its integrand is . The Euler-Lagrange equation and Beltrami identity become
As tends to zero at an endpoint, also tends to zero, so . On the positive interior this implies and, writing ,
The Euler-Lagrange equation also yields . Therefore
including at the maximum where . Integrating and using the first integral gives
Thus the regular extremal is a sphere. The two zeros fix its center and radius:
Equivalently, the prescribed volume fixes . If are fixed in advance, their separation and the prescribed volume must be compatible with these equations; there is no positive regular stationary profile of the stated type for an incompatible volume.
For minimality, the three-dimensional isoperimetric inequality states that every sufficiently regular closed surface enclosing volume has
with equality precisely for a sphere. Our extremal has and , so it attains equality and is a global minimum of area at the fixed volume. This also makes clear the connected-surface assumption behind uniqueness: allowing several disconnected components introduces stationary collections of spheres, rather than the single positive generating profile used here.
Ignoring a factor independent of the parameters, the likelihood function is
Since every count is positive, a boundary value has zero likelihood. At an interior optimum, a Lagrange multiplier for gives . Summing the resulting gives , so
The log likelihood function is strictly concave on the positive simplex, because its Hessian matrix is diagonal with entries . Hence this critical point is the unique global maximum-likelihood estimate.
Under independence, the log likelihood function separates:
Maximizing the two terms over their respective probability simplices, by the same Lagrange multiplier argument as in part (a), gives
Therefore the constrained maximum-likelihood estimates are
The associated fitted cell counts are .
Take a smooth variation with on the boundary. Differentiating under the integral gives the first variation
Integration by parts removes the derivatives of , and the boundary contribution vanishes. The fundamental lemma of the calculus of variations therefore gives
For a regular constraint , the Lagrange multiplier rule says that an extremal satisfies for a constant . Equivalently, apply the preceding Euler–Lagrange equation to . Regularity means that is not identically zero as a linear functional on the admissible variations. For the quadratic constraint below this holds at every nonzero admissible ; a zero constraint leaves only .
In polar coordinates the constrained integrand is
Its Euler–Lagrange equation is , or
The field is single-valued and periodic in . Substitution with gives
The centre is part of the disk, so one must also select the regular radial solution: for , smoothness gives and the regular derivative behaviour; for , is bounded and . A singular solution at zero does not define an admissible smooth field on the disk.
For fixed , let , so and for . The angular derivative term has integral , including zero when . Consequently
The one-dimensional Euler–Lagrange equation for is
exactly the same radial equation. The common factor cancels.
For two regular solutions with the same and different multipliers, multiply the equation for by , multiply that for by , subtract and integrate. The result is
At , both functions vanish; at zero, regularity makes the boundary term tend to zero. Since the multipliers are distinct,
This is weighted Sturm–Liouville orthogonality with weight .
Finally, multiply one radial equation by and integrate by parts. The same endpoint conditions yield
Multiplication by proves
For any nonzero admissible solution the energy on the left before subtraction is positive, so its multiplier is the positive Rayleigh quotient .
On the constraint set, , so . Part (a)(iv) and the Lagrange multiplier theorem give , hence
Since is nonzero, it is an eigenvector. Moreover , so the associated eigenvalue is real and nonnegative.
The Lagrange multiplier theorem for one regular constraint says: if are near , , , and is a local extremum of restricted to , then
For several constraints with linearly independent gradients, the objective gradient lies in their linear span. The regularity assumption rules out singular points where the usual multiplier conclusion can fail.
Ignoring a factor independent of the probabilities, the likelihood function is , so its log is subject to . A Lagrange multiplier gives . Summing makes .
Thus
Since all counts are positive, a zero fitted cell probability would have zero likelihood. On the interior the log-likelihood is strictly concave, so this stationary point is its unique global maximum.
On the constraint set, , so . Part (a)(iv) and the Lagrange multiplier theorem give , hence
Since is nonzero, it is an eigenvector. Moreover , so the associated eigenvalue is real and nonnegative.