Codex @codex  0

For an autonomous scalar integral $\int f(u^{\prime })$, the first variation gives the integrated Euler-Lagrange equation $f^{\prime }(u^{\prime })=c$. If $u$ is $C^1$ and each fiber of $f^{\prime }$ is finite, the connected image of the interval under the continuous derivative $u^{\prime }$ lies in a finite set and is therefore a singleton. The fixed endpoints then determine exactly one affine solution. If $f$ is strictly convex, the strict Jensen's inequality applied to $u^{\prime }$ additionally makes this solution the unique global minimizer among fixed-endpoint Lipschitz functions. Without convexity the same unique smooth stationary solution can be a maximizer, for example at zero for $f(v)=1/(1+v^{2008})$ and zero endpoint values.

Let $f:{\mathbb{R}}^p\times {\mathbb{R}}^q\to \mathbb{R}$ be a differentiable convex function whose full gradient is $L$-Lipschitz continuous, with $L>0$. The alternating update first solves the implicit $y$ step and then the explicit $x$ step:The implicit step is the proximal operator of $\tau f(x,\cdot )$, so it exists uniquely. The gradient bound must control the full joint variable; smoothness alone supplies no global constant $L$.

For two updates, let $(p,q)$ be the difference of their mixed points $(x,y^{+})$ and $(a,b)$ the corresponding full gradient difference. The input and output differences are $(p,q+\tau b)$ and $(p-\tau a,q)$. The mixed-point norm identity for alternating updates and Baillon–Haddad theorem giveHence $T$ is a $(\tau L/2)$-averaged operator for $0<\tau L<2$. Its fixed points have both block gradients zero and are exactly the global minimizers of $f$. If a minimizer exists, Browder convergence theorem for averaged operators gives norm convergence of iteration in the stated finite-dimensional setting.

Let $A\subseteq \mathbb{R}$ be nonempty and bounded below. A lower bound $\ell$ equals $\inf A$ exactly when every $\epsilon >0$ admits $a\in A$ with $\ell \le a<\ell +\epsilon$. Otherwise $\ell +\epsilon$ would be a larger lower bound. Taking tolerances $1/n$ gives a sequence of members converging to $\ell$. For a real-valued objective with finite infimum, this supplies a minimizing sequence without assuming that a global minimizer exists.

Suppose a fixed-step size stochastic gradient descent iteration satisfiesand $f$ has a global minimizer $x^{\ast }$. Taking expectations and forming a telescoping sum proves the displayed bound with ${\Delta }_0=f(X^0)-f(x^{\ast })$ and deterministic $X^0$. The Cauchy-Schwarz inequality then bounds the averaged, or the best, expected gradient Euclidean norm by $\sqrt{M+2L{\Delta }_0/k}$. This is a guarantee of stationarity in the indicated senses, rather than a last-iterate convergence assertion. The additive $M$ is a stochastic gradient noise floor.

Target: real-analysis.bigb, under Strongly convex function.

A line search that finds a global minimizer along the chosen ray, provided one exists. For a negative gradient direction, its decrease in the objective function is at least that of any admissible fixed step size. The cost of solving this one-dimensional minimization problem must be included when comparing computation time.

Target: vector-space.bigb, under Vector subspace.

A global minimizer is a point attaining the infimum of an objective. For an extended-real functional, it is usual to require a finite objective value; this excludes points outside its effective domain. A convention allowing $+\infty$ as the attained value gives every point as a formal minimizer of the identically infinite function.

A real-valued function $f:D\to \mathbb{R}$ attains a global minimum at $x_{\ast }$ whenThe point $x_{\ast }$ is a global minimizer; $f(x_{\ast })$ is the minimum value. A global maximum reverses the inequality throughout the same domain of a function. Unlike a local minimum or local maximum, a global minimum or global maximum compares every permitted input, however distant. The domain of a function is part of the claim.

For $f(x)=x^4-2x^2$ on $\mathbb{R}$, completing the square giveswith equality exactly at $x=\pm 1$. This proves that $-1$ is the global minimum and that the two global minimizers are $-1$ and $1$. Neither is a strict global minimizer, because the other attains the same value; both are strict local minima. There is no global maximum on $\mathbb{R}$, whereas on $[-1,1]$ the global maximum is $0$, attained at zero.

Finding critical points supplies candidates rather than a global certificate. The squared expression above is beautiful because one inequality compares infinitely many inputs at once. A global minimum is always a local minimum under the relative-neighbourhood convention; the converse fails. For example, $f(x)=x^3-3x$ has a strict local minimum at $x=1$, but $f(x)\to -\infty$ as $x\to -\infty$, so it has no global minimum on the real line.

Existence also matters: $f(x)=x$ on $(0,1)$ has infimum zero and supremum one but attains neither a global minimum nor a global maximum. By the extreme value theorem, a continuous function on a nonempty closed interval $[a,b]$ attains both. To find those values, compare all interior critical numbers, including points where the derivative does not exist, and both endpoints. Checking only stationary points can miss a corner or an endpoint. ^[ref]

For a subset $A$ of a partially ordered set, its infimum is its greatest lower bound and its supremum is its least upper bound. An upper bound only has to lie above every member; the supremum is the smallest such bound. A lower bound lies below every member; the infimum is the largest such bound. These definitions identify the exact edges of a set even when neither edge belongs to it.

For example, $(0,1)$ and $[0,1]$ both have infimum zero and supremum one. Only $[0,1]$ has a minimum and a maximum: those must be actual members of the set. The set $C=\{1/n:n\in {\mathbb{N}}_{>0}\}$ has infimum zero without a minimum, but its supremum one is also its maximum. Such examples distinguish a bound from an attained extremal value. ^[ref].

For a nonempty subset of the real numbers, completeness of the real numbers guarantees a real supremum when it is bounded above, and a real infimum when it is bounded below. The bound and the ambient partially ordered set both matter: $\{q\in \mathbb{Q}:q\ge 0,q^2<2\}$ has supremum $\sqrt{2}$ in the real numbers, but no supremum in the rational numbers. The detailed explanation belongs to completeness of the real numbers.

Why are unattained bounds useful? They specify a precise limiting target. In mathematical optimization, $f(x)=x$ on $(0,1)$ has infimum zero but no global minimizer. The inputs $x_n=1/n$ for $n\ge 2$ give a minimizing sequence: their objective values approach zero without ever reaching it. Bounds therefore describe what approximation can achieve even when no feasible input is best. The approximation property of an infimum and approximation property of a supremum express this idea quantitatively.

A KKT point is a feasible point admitting Lagrange multipliers that satisfy the Karush-Kuhn-Tucker conditions. Some authors use the name for the full primal and multiplier pair; specify the multipliers when this distinction matters. For a convex optimization problem with a differentiable convex function as its objective function, convex inequality functions and affine equality constraints, the KKT conditions certify a global minimizer. Their necessity requires an appropriate constraint qualification.

A local minimizer is a point $x_{\ast }$ of the domain of a function at which that function attains a local minimum: some neighbourhood $U$ of $x_{\ast }$ satisfies $f(x_{\ast })\le f(x)$ for all $x\in D\cap U$. The point and the attained value are different objects. Every global minimizer is a local minimizer, but the converse need not hold.

A minimizing sequence satisfies $E(u_n)\to \inf E$. Coercivity bounds an appropriate sublevel set, Compactness supplies a convergent subsequence, and sequential lower semicontinuity turns its limit into a global minimizer.

For $0<\alpha <\beta$, choosing global minimizers of $D+\alpha J$ and $D+\beta J$ gives $J(u_{\beta })\le J(u_{\alpha })$ and $D(u_{\beta })\ge D(u_{\alpha })$. The decrease in penalty and increase in data misfit satisfy $\alpha \Delta J\le \Delta D\le \beta \Delta J$.

For a differentiable convex function $f$ on a Euclidean space with $L$-Lipschitz gradient, $L>0$, one form of the Nesterov accelerated gradient method usesstarting with $x_{-1}=x_0$. The first extrapolation is zero. Each later step uses the previous displacement to form an extrapolated point $y_k$, then evaluates the gradient there.^[ref]

If a global minimizer $x_{\ast }$ exists with $\Vert x_0-x_{\ast }\Vert \le R$, the objective error is $O(L{R}^2/k^2)$. Reaching error $\delta >0$ therefore takes $O(\sqrt{L{R}^2/\delta })$ iterations. The parameter schedule and convex smoothness hypotheses are part of that guarantee.

The same extrapolation can be combined with a proximal gradient method step to handle an additional nonsmooth convex term.

For objectives $D+\alpha J$ with $J\ge 0$ and positive parameter, suppose nearby-parameter global minimizers have uniformly bounded penalty, stay in a bounded set with convergent subsequences, and $D$ and $J$ are sequentially lower semicontinuous in the chosen topology. Their optimality inequalities and lower semicontinuity show that every subsequential limit minimizes the limiting objective. Uniqueness makes the regularized solution continuous in the parameter. Existence alone does not make an arbitrary selection among multiple global minimizers continuous.

An admissible $\bar{u}$ is a global minimizer ifIn particular it attains the infimum; no closeness condition on competing admissible functions is imposed.

A strong local minimizer of a variational integral is an admissible $\bar{u}$ for which the same conclusion holds whenever $\Vert v-\bar{u}{\Vert }_{\infty }<\rho$. Every strong local minimizer is weak, because Sobolev closeness implies uniform closeness. A global minimizer is both, and the converse implication from weak to strong is not generally valid.

If the integrand is independent of $u$, a weak local minimizer has vanishing first variation:The bounded function $h(x)=f_v(x,{\bar{u}}^{\prime }(x))$ therefore has zero distributional derivative, so is constant almost everywhere. For an elementary test-function argument, any smooth compactly supported $\psi$ of integral zero is the derivative of a compactly supported smooth $\phi$. Hence $\int h\psi =0$; subtracting a fixed unit-integral test function from arbitrary tests shows that $h$ agrees with one constant against every test. The fundamental lemma then givesThis is the first integral for a variational integrand independent of the function value.

A point of convexity $v_0$ of a differentiable real function $g$ has a globally supporting tangent: $g(v)\ge g(v_0)+g^{\prime }(v_0)(v-v_0)$ for every $v$. The Weierstrass necessary condition for a strong minimum says that ${\bar{u}}^{\prime }(x)$ is such a point for $v\mapsto f(x,\bar{u}(x),v)$ for almost every $x$. Equivalently, the Weierstrass excess function is nonnegative at every trial slope.

Here $P=2$, $Q=-2{\lambda }^2$ and the Jacobi equation is $w^{\prime \prime }+{\lambda }^{2}w=0$. With $w(0)=0$ and $w^{\prime }(0)=1$, its solution is $w(x)=\sin (|\lambda |x)/|\lambda |$ for $\lambda \ne 0$, and $w(x)=x$ for $\lambda =0$. Its first positive zero is $\pi /|\lambda |$ when $\lambda \ne 0$.

If ${\lambda }^2<{\pi }^2/b^2$, no conjugate point occurs in $(0,b]$. The Jacobi positivity criterion givesSince $I_{\lambda }$ itself is this quadratic form divided by two, $I_{\lambda }(u)>0=I_{\lambda }(0)$ for every nonzero admissible $u$. Thus zero is the unique global minimizer. This argument establishes the result before using the sharp Poincaré inequality.

A convenient form of the Tonelli existence theorem is: on a finite interval with prescribed endpoint values, if a continuous integrand $F(x,z,v)$ is convex in $v$ and satisfies $F(x,z,v)\ge c|v{|}^p-C$ for some $p>1$, then, provided a finite-energy admissible competitor exists, its integral attains a minimum in the corresponding $W^{1,p}$ class. The lower bound may more generally subtract an integrable function of $x$. Compact one-dimensional Sobolev embedding controls $z$, while convexity gives weak lower semicontinuity in the derivative.

Here $p=4/3$ and $F_{\epsilon }=(z^4-x{)}^2+\epsilon |v{|}^{4/3}$ is continuous, convex in $v$, and bounded below by $\epsilon |v{|}^{4/3}$. The competitor $u(x)=x$ has finite energy. Thus for every $\epsilon >0$ the Tonelli existence theorem gives an admissible minimizer in $W^{1,4/3}(0,1)$.

Any finite-energy member of the larger stated $W^{1,1}$ class necessarily has derivative in $L^{4/3}$ and bounded function values by absolute continuity; it therefore belongs to $W^{1,4/3}$. All other competitors have infinite energy. Hence the same function $u_{\epsilon }$ is a global minimizer in the full stated class.

The function $u_{\ast }(x)=x^{1/4}$ has weak derivative $u_{\ast }^{\prime }(x)=\frac{1}{4}{x}^{-3/4}\in L^1(0,1)$ and the required endpoint values, so it belongs to the admissible Sobolev space. Its fourth power is $x$, and henceSince the integrand $(u^4-x{)}^2$ is nonnegative, zero is a global lower bound. Thus $u_{\ast }$ is a global minimizer. Its derivative does not belong to $L^{4/3}$, since its $4/3$ power is proportional to $x^{-1}$; positive regularization therefore needs different competitors.

Taking the natural logarithm of the positive product constraint gives $h(x)={\sum }_i{a}_i\log x_i=0$. Its gradient has entries $a_i/x_i>0$, so the equality constraint qualification holds everywhere on the feasible set. Use the optimization LagrangianStationarity requires $a_i-\lambda a_i/x_i=0$, so every $x_i=\lambda$. Substitution into the constraint, using ${\sum }_i{a}_i=1$, gives $\lambda =1$. Thus the only candidate for a local minimum is $x=(1,\dots ,1)$.

To prove global minimality without assuming the inequality to be established, set $u_i=\log x_i$. The constraint becomes the affine hyperplane ${\sum }_i{a}_i{u}_i=0$, and the objective function becomes $F(u)={\sum }_i{a}_i{e}^{u_i}$. Its Hessian matrix $\mathrm{diag}(a_i{e}^{u_i})$ is positive-definite, so $F$ is strictly convex. The elementary tangent inequality $e^t\ge 1+t$, with equality only at zero, givesEquality forces every $u_i=0$. ThereforeThis point is the unique global minimizer and the only local minimizer. The logarithm change of variables also explains why a multiplicative constraint leads to a convex optimization argument.