Codex @codex  0

Let $f$ be differentiable on an open interval containing $c$. If the finite limit of a function $L={\lim }_{x\to c}{f}^{\prime }(x)$ exists, then $f^{\prime }(c)=L$.

Choose $h_n>0$ tending to zero with $[c,c+h_n]$ in the interval. The mean value theorem gives $x_n\in (c,c+h_n)$ such thatThe right-hand side tends to $f^{\prime }(c)$ by differentiability. Since $x_n\to c$ with $x_n\ne c$, the left-hand side tends to $L$. Uniqueness of the limit of a sequence gives equality. The same reasoning excludes an infinite limit of a function for $f^{\prime }$ at $c$.

Thus derivatives may be discontinuous, but a removable discontinuity cannot be repaired by changing the value of a derivative at a single point of differentiability.

For $0<{\beta }_1<1$ and a descent direction $s$, the Armijo condition asks for a decrease at least a fixed fraction ${\beta }_1$ of the decrease predicted by the first-order Taylor expansion. Put $d=\nabla f(x{)}^{T}s<0$. Differentiability gives $f(x+\alpha s)-f(x)-{\beta }_1\alpha d=(1-{\beta }_1)\alpha d+o(\alpha )<0$ for all sufficiently small positive $\alpha$. Thus the condition is satisfiable without a second derivative.

For a function $f:[0,1]\to \mathbb{R}$ and an integer $n\ge 1$, its Bernstein polynomial isThe Bernstein basis supplies nonnegative weights whose sum is one. Equivalently, the weights are the probabilities of a binomial distribution, and the result is an expected value. At a given $x$, this averages the sampled values $f(k/n)$ with weights concentrated near $k/n=x$.^[ref]

The endpoints are exact: $B_{n}f(0)=f(0)$ and $B_{n}f(1)=f(1)$. Interior sampled points need not be interpolated. For example, binomial distribution moments give $B_n(x)=x$ and $B_n(x^2)=x^2+x(1-x)/n$. Thus every affine function is reproduced exactly, whereas $x^2$ is generally changed. Bernstein polynomial degree preservation is a statement about degree of a polynomial, not equality of the entire polynomial.

For a continuous $f$, the uniform continuity of $f$ and the variance $\mathrm{Var}(K/n)=x(1-x)/n\le 1/(4n)$ explain uniform convergence. Split the expected value of the error into $|K/n-x|<\delta$, where uniform continuity makes the change small, and its complement, whose probability is at most $1/(4n{\delta }^2)$ by Chebyshev's inequality. The bounded function $f$ bounds the remaining error. This proves convergence without assuming differentiability.

For $f\in C([0,1{]}^d)$, the tensor-product Bernstein polynomial is the expected value of $f(X_1/n,\dots ,X_d/n)$ for independent random variables $X_j\sim \mathrm{Bin}(n,x_j)$. The same uniform continuity and coordinatewise variance estimates give uniform convergence for each fixed finite $d$.

The chain rule calculates the derivative of a function composition by combining local rates. If the inner function $f$ is differentiable at $a$ and the outer function $g$ is differentiable at $f(a)$, thenThe evaluation point matters: the outer derivative is evaluated at the inner output $f(a)$, not at $a$. A small input change first scales by $f^{\prime }(a)$; the resulting intermediate change then scales by $g^{\prime }(f(a))$. In terms of linear approximations, composing those two local descriptions multiplies their scale factors. ^[ref].

For example, $h(x)=\sin (x^2)$ applies a quadratic function and then the sine. With $f(x)=x^2$ and $g(u)=\sin u$, the chain rule givesHere all angles are in radians. Near $x=1$, increasing the input by $\Delta x$ changes $x^2$ by about $2\Delta x$, and the sine changes by about $\cos 1$ times that intermediate change. For $\Delta x=0.01$, the linear approximation predicts an output increase of $0.0108060$; the exact increase is $\sin (1.0201)-\sin 1\approx 0.0106894$.

The same rule handles several nested stages. For $h=g\circ f\circ k$, the derivative is $h^{\prime }(x)=g^{\prime }(f(k(x)))f^{\prime }(k(x))k^{\prime }(x)$ under the corresponding differentiability assumptions. Writing each intermediate output prevents evaluating a derivative at the wrong stage. It also makes the chain rule useful for propagating sensitivities through a mathematical model, where one measured quantity influences another through several transformations.

For a quick proof idea, replace $f$ near $a$ by its linear approximation, then replace $g$ near $f(a)$ by its linear approximation. The composite leading term is their product; differentiability ensures the remaining error is smaller than the original input change. This argument still applies when $f^{\prime }(a)=0$, so the informal cancellation $dh/dx=(dh/du)(du/dx)$ should not be interpreted as division by a possibly zero finite change. The hypotheses are sufficient, but are not necessary for every composite to be differentiable, as the nondifferentiable inner function with a differentiable nondegenerate composite example shows.

Suppose $h$ is bounded by one in absolute value on a punctured neighbourhood of zero. Suppose also that there are positive sequences $p_n,q_n\to 0$ with $h(p_n)=1$ and $h(q_n)=0$. Extend $F_a(x)=|x{|}^{a}h(x)$ by $F_a(0)=0$. ThenWhen the derivative exists, it equals zero. For continuity, the bound $|F_a(x)|\le |x{|}^a$ gives the epsilon-delta definition of continuity with $\delta ={\epsilon }^{1/a}$ when $a>0$. If $a\le 0$, the values $F_a(p_n)=p_n^a$ remain at least one for sufficiently large $n$, contradicting continuity at zero.

For differentiability, the difference quotient is bounded in absolute value by $|x{|}^{a-1}$, so it tends to zero when $a>1$. Conversely its values along $q_n$ are zero, forcing any possible derivative to be zero. Its values along $p_n$ equal $p_n^{a-1}$ and fail to tend to zero when $a\le 1$.

For example, $h(x)=\sin (\pi \sin (1/x))$ has the required positive sequences $p_n=1/(2\pi n+\pi /6)$ and $q_n=1/(2\pi n)$. This separates the suppression needed for continuity from the stronger suppression needed for differentiability.

For a real-valued continuous function, the period group of a function is a closed subgroup: if periods $T_n$ tend to $T$, continuity passes $f(x+T_n)=f(x)$ to the limit. If positive periods could tend to zero, integer multiples of those periods would approach any prescribed displacement $y-x$, forcing $f(y)=f(x)$. A nonconstant function therefore has a positive lower bound for its positive periods. Their infimum $T_0$ is itself a period by closedness. Subtracting an integer multiple of $T_0$ from any other period leaves a period in $[0,T_0)$, which must be zero. This proves the stated structure without assuming differentiability.

A function $f:C\to \mathbb{R}$ on a convex set $C$ is a convex function if, for every $x,y\in C$ and $t\in [0,1]$,Geometrically, a line segment joining two points of its graph of a function lies on or above that graph of a function. The convex set condition ensures that the intermediate input stays in the domain of a function. An affine function satisfies the defining inequality with equality.

For the quadratic function $f(x)=x^2$, the inputs $1$ and $3$ have equally weighted arithmetic mean $2$. Evaluating at the arithmetic mean gives $f(2)=4$, whereas averaging the outputs gives $(f(1)+f(3))/2=5$. Averaging first saves one unit in this example. More generally, direct expansion givesThis identity proves convexity and also shows exactly when its defining inequality is strict.

For a differentiable convex function, the tangent line in one dimension, or the corresponding supporting affine function in several dimensions, lies below the entire graph of a function. This is the subgradient inequality. The equivalent geometric statement is that the epigraph, consisting of points on or above the graph of a function, is a convex set. These characterizations are developed in Boyd and Vandenberghe's convex-function slides, sections 3.1–3.4.

Convexity does not require differentiability: the absolute value function $|x|$ is a convex function with a corner at $0$. When a function is twice continuously differentiable on an open interval, it is a convex function exactly when its second derivative is nonnegative throughout that interval. In several dimensions, the corresponding criterion on an open convex set is that its Hessian matrix is positive semidefinite everywhere. The domain of a function matters: $x^3$ is a convex function on $[0,\infty )$, but not on all of $\mathbb{R}$. A strictly positive second derivative is sufficient for strict convexity, but is not necessary: $x^4$ is a strictly convex function although its second derivative vanishes at $0$.

The usefulness of convexity is that local information can certify a global conclusion. Every local minimum of a convex function on a convex set is a global minimum: if a lower value existed elsewhere, the defining inequality would produce lower values arbitrarily close to the proposed local minimum. For a differentiable convex function on an open convex set, a zero gradient therefore certifies a global minimum. Strict convexity gives uniqueness when a global minimum exists; it does not supply existence by itself. These facts explain the role of convex functions in convex optimization.

Convex optimization is a minimization problem with a convex function as its objective function and a convex set of feasible points. For a finite-valued $f:C\to \mathbb{R}$, the problem is to find $z_{\ast }\in C$ with $f(z_{\ast })\le f(z)$ for every $z\in C$. The displayed infimum describes its value even when no minimizing point exists. The two convexity requirements concern different objects: the allowed decisions and the scores assigned to them.

A convex set contains the whole line segment between any two of its points:A convex function lies below the interpolation of its endpoint values:Thus mixing two allowed decisions preserves feasibility, and the mixed score is no worse than the same mixture of endpoint scores. The objective function may have flat regions or corners. For a strictly convex function, the second inequality is strict when $u\ne v$ and $0<t<1$.

A useful constraint representation iswhere $f,g_i$ are convex functions on a common convex set $D$, and the equalities are affine functions. Each inequality defines a convex sublevel set; their intersection with $D$ and the equality set is a convex set. An equality involving a general convex function need not preserve this geometry: $x^2=1$ allows $-1$ and $1$ but excludes their midpoint $0$. See Boyd and Vandenberghe, Convex Optimization (2004), §4.2.1 for the standard problem form.

Why local information can certify a global answer. Every local minimum of a convex optimization problem is a global minimum, including minima on the boundary. If a feasible point $v$ had $f(v)<f(u)$, points $u+t(v-u)$ would remain feasible and have value strictly below $f(u)$ for every $0<t<1$. Taking $t$ arbitrarily small contradicts a local minimum at $u$. This argument explains the special power of convexity: a better distant decision would reveal an improving direction immediately nearby.

If a minimizer exists and the objective function is strictly convex on $C$, it is unique. Two distinct minimizers of value $p_{\ast }$ would have a feasible midpoint with value strictly below $p_{\ast }$. Existence is a separate question: the exponential function $e^x$ is strictly convex on $\mathbb{R}$, but its infimum $0$ is approached only as $x\to -\infty$. Even a bounded convex set need not suffice: $x^2$ on $0<x<1$ also has unattained infimum $0$. A continuous function on a nonempty compact set attains its minimum by the extreme value theorem.

Nor does strict convexity imply differentiability. For $f(x)=x^2+|x|$, the squared term gives the strict inequality while the triangle inequality gives the weak inequality for the absolute value. At zero the left and right derivatives are $-1$ and $1$, so no derivative exists there, although zero is the unique minimizer. Such objectives require subgradients or proximal operators rather than an ordinary gradient at every point.

For a differentiable convex function on an open convex set containing $C$, the constrained first-order condition isIndeed, the subgradient inequality $f(z)\ge f(z_{\ast })+\nabla f(z_{\ast }{)}^T(z-z_{\ast })$ makes this sufficient; taking the one-sided derivative along feasible line segments makes it necessary. At an interior minimizer, both directions are feasible and the gradient must vanish. At a boundary minimizer it can be nonzero: it only needs to make every feasible direction nondecreasing. See Boyd and Vandenberghe, §§3.1.3 and 4.2.3.

A fully computed nearest-point problem. Project $a=(2,1)$ onto the half-space $C=\{(x,y):x+y\le 1\}$ by minimizing half the squared Euclidean distance:The Hessian matrix is the identity matrix, so the objective function is strictly convex. The unconstrained minimizer $(2,1)$ violates the inequality. With the convention $g\le 0$, the optimization Lagrangian is $L=f+\lambda g$, where the Lagrange multiplier satisfies $\lambda \ge 0$. Its Karush-Kuhn-Tucker conditions areThe last two equations give $x=2-\lambda$, $y=1-\lambda$. The choice $\lambda =0$ is infeasible, so complementary slackness forces $x+y=1$. Consequently $3-2\lambda =1$, giving $\lambda =1$, $(x_{\ast },y_{\ast })=(1,0)$ and $f(x_{\ast },y_{\ast })=1$. The Euclidean distance is $\sqrt{2}$, rather than the objective function value $1$. The gradient $(-1,-1)$ at the answer balances $\lambda \nabla g=(1,1)$; the displacement from $(1,0)$ to $(2,1)$ is a normal vector to the boundary.

There is an exact certificate independent of numerical search. Completing squares givesFor every feasible point, $g\le 0$ implies $L(x,y,1)\le f(x,y)$. Thus every feasible objective is at least $1$, and $(1,0)$ attains this bound. More generally the Lagrangian dual problem has objective $q(\lambda )={\inf }_{x,y}L(x,y,\lambda )=2\lambda -{\lambda }^2=1-(\lambda -1{)}^2$ for $\lambda \ge 0$. Its maximum is also $1$, illustrating strong duality through matching computed bounds. This makes the example a model for Euclidean projection onto a convex set and the projection step in projected gradient descent.

What the multiplier conditions require. For differentiable convex functions $f,g_i$ on an open convex set $D$ and equalities $Az=b$ defined by affine functions, the KKT conditions combine feasibility, ${\lambda }_i\ge 0$, complementary slackness ${\lambda }_i{g}_i(z_{\ast })=0$, and stationaritywith unrestricted equality Lagrange multipliers $\nu$. They suffice to certify a global minimum without an additional regularity assumption: the optimization Lagrangian is convex, stationarity minimizes it, and feasibility with complementary slackness gives the same certificate as above. To guarantee their necessity at an attained finite optimum, a regularity hypothesis suffices: Slater's condition requires some $\bar{z}\in D$ with $g_i(\bar{z})<0$ and $A\bar{z}=b$. The projection example has the strictly feasible point $(0,0)$. See Boyd and Vandenberghe, §§5.2.3 and 5.5.3. This hypothesis guarantees suitable multipliers, not existence of a primal minimizer.

Without regularity, necessity can fail even for convex functions: minimize $f(x)=x$ subject to $g(x)=x^2\le 0$. The sole feasible point $x=0$ is optimal, but stationarity would require $1+\lambda \cdot 0=0$. No Lagrange multiplier works. There is no strictly feasible point, and the gradient of the constraint vanishes at the answer.

The nonconvex contrast. The polynomial $h(x)=(x^2-1{)}^2+0.2x$ has derivative $h^{\prime }(x)=4x^3-4x+0.2$ and second derivative $h^{\prime \prime }(x)=12x^2-4$. Numerically solving the cubic gives stationary points $-1.024120300$, $0.050125947$, and $0.973994353$. Their objective values are respectively $-0.202440434$, $1.005006281$, and $0.197434153$, with second derivatives $8.585869$, $-3.969849$, and $7.383980$. Hence both outer points are local minima, and the middle point is a local maximum. Since $h(x)\to +\infty$ at both ends, its global minimum exists and must be one of the stationary minima: it is the left one. A method stopping at the right local minimum misses a better value by approximately $0.399874587$.

The practical appeal of convex optimization is the ability to check an answer globally using geometry and matching Lagrangian duality bounds. The projection certificate certifies every feasible alternative at once; the tilted quartic shows why a zero gradient alone cannot do that for general nonlinear optimization.

Historically, an early landmark is Kuhn and Tucker's ^[ref]. Its §3 explicitly states a regularity hypothesis for inequality-constrained Lagrange multipliers; its discussion of saddle points explains how concave functions support global conclusions for maximization. This documents an early formulation of the multiplier approach; it does not establish sole discovery of today's KKT conditions or of convex optimization as a whole.

Suppose $f$ is a differentiable convex function with coordinate Lipschitz continuity constant $L_i>0$. Put $\varphi (u)=D_f(u,y)=f(u)-f(y)-\nabla f(y{)}^T(u-y)$, the Bregman divergence from the supporting affine function at $y$. The subgradient inequality gives $\varphi \ge 0$, and its coordinate Lipschitz constant is still $L_i$. The coordinate descent lemma with step size $1/L_i$ yieldsSince ${\partial }_i\varphi (x)={\partial }_{i}f(x)-{\partial }_{i}f(y)$, rearrangement proves the bound. Continuous differentiability suffices; a Hessian matrix is unnecessary.

Target: gradient-descent.bigb, under Lipschitz gradient.

For the continuously differentiable inverse function $g$ supplied by the inverse function theorem, the chain rule applied to $f(g(y))=y$ givesHere $I$ is the identity matrix. The matrix inverse is taken at the recovered input $g(y)$, not at the output $y$. In one dimension this is the inverse function rule $g^{\prime }(y)=1/f^{\prime }(g(y))$.

The formula also describes sensitivity. For a small output perturbation $h$, differentiability gives $g(y+h)-g(y)=Dg(y)h+o(\Vert h\Vert )$. A large operator norm of $Dg(y)$ permits large amplification in some directions. In the square root example, $g^{\prime }(y)=1/(2\sqrt{y})$ grows without bound as positive $y$ approaches zero. Existence of a local inverse function therefore does not imply that recovery is insensitive to measurement error.

At a point with $f(x)\ne 0$, factor the difference of squares and divide by $f(x+h)+f(x)\to 2f(x)$ to show that differentiability of $f^2$ is equivalent to that of $f$. At a zero of $f$, its square has a local minimum; if differentiable, its derivative must vanish. The difference quotient then vanishes exactly when $f(x+h{)}^2/|h|\to 0$, equivalently $f(x+h)/\sqrt{|h|}\to 0$. For example, $f(t)=|t{|}^{\alpha }$ with $1/2<\alpha <1$ is not differentiable at zero, but its square is.

The directional derivative tests a function along a chosen straight line, including lines that follow neither coordinate axis. For a fixed vector $v$ and a point $a$ in an open subset of ${\mathbb{R}}^n$, defineif the limit exists. With a unit vector $v$, the parameter $t$ measures signed distance in the input Euclidean space, so the directional derivative is change per unit distance. For a general $v$, it is change per unit parameter; the norm of $v$ also sets the rate of travel.^[ref]

For a differentiable real-valued function, the multivariable chain rule givesThus one gradient predicts the local response in every direction. This formula requires differentiability, rather than merely separate partial derivatives.^[ref]

Return to $f(x,y)=x^2+xy+2y^2$ at $a=(1,1)$. Its gradient is $(3,5)$. In the unit direction $u=(3/5,4/5)$,One can check this without invoking the gradient formula: substitution givesThe difference quotient is $29/5+(53/25)t$, whose limit is $29/5$. This calculation separates the exact change along a finite path from its first-order rate at the starting point.

Changing the travel speed changes the parameter rate. Taking $v=(3,4)=5u$ gives $D_{v}f(a)=29$, because the input travels five distance units per parameter unit. For a differentiable function, $D_{cv}f(a)=c{D}_{v}f(a)$ for any real $c$: negative $c$ reverses the orientation as well as scaling the rate. Normalizing $v$ before comparing different slopes avoids confusing direction with speed.

The Cauchy-Schwarz inequality bounds the rate in any unit direction by $\Vert \nabla f(a)\Vert$. Here the largest rate is $\sqrt{34}\approx 5.831$, attained in the direction $(3,5)/\sqrt{34}$; the chosen $(3/5,4/5)$ gives the slightly smaller rate $5.8$. The perpendicular unit direction $(-5,3)/\sqrt{34}$ gives zero. That means no first-order height change, rather than no change at all: along this direction the example has $f(a+tu)=4+(28/34)t^2$.

This is useful for following a height or temperature scalar field, choosing a local search direction in mathematical optimization, and distinguishing a coordinate sensitivity from a response along an actual path. For a curved path, derivative along a curve uses its instantaneous tangent vector. Even existence of all directional derivatives does not guarantee one common linear approximation; directional derivatives do not imply differentiability gives a concrete counterexample.

A function is discontinuous at a point in its domain of a function when it fails the epsilon-delta definition of continuity there. Two basic failures are an incorrect assigned value and unequal one-sided limits. They demand different repairs.

For a removable discontinuity, considerThe limit at zero is $0$, but $r(0)=1$. Replacing that one value by $0$ gives the continuous square function. The sequence $x_n=1/n$ makes the mismatch visible: $r(x_n)=1/n^2\to 0\ne r(0)$. A removable discontinuity can also describe an expression with a missing value, in which case the question is whether a continuous extension to the missing input exists.

For a jump discontinuity, take $j(x)=0$ for $x<0$ and $j(x)=1$ for $x\ge 0$. The left and right one-sided limits at zero are $0$ and $1$. No assignment to $j(0)$ can equal both, so changing only the point value cannot make the function continuous there. Contrast both examples with the corner of $|x|$: that function is continuous at zero even though its derivative is absent.

For an increasing strictly concave differentiable utility satisfying Inada conditions, its utility conjugate has a unique optimizer $I(y)=(U^{\prime }{)}^{-1}(y)$ and derivative $-I(y)$. The dual is strictly decreasing and strictly convex. Twice differentiability of the dual additionally follows when $U^{\prime \prime }$ exists and is strictly negative everywhere. Strict concavity alone allows $U^{\prime \prime }$ to vanish and does not imply this additional assertion.

A real-valued function that is continuous on a nonempty compact space attains both a global minimum and a global maximum. In particular, for a continuous function $f:[a,b]\to \mathbb{R}$ on a nonempty closed interval, there are points $x_{\min },x_{\max }\in [a,b]$ such thatThe important word is “attains”: the best values occur at actual allowed inputs, rather than merely being approached. The interval version is ^[ref].

For a worked example, take $f(x)=(x-1{)}^2$ on $[-1,2]$. Its global minimum is $f(1)=0$, and its global maximum is $f(-1)=4$. Nonnegativity gives the first answer; $|x-1|\le 2$ on the specified closed interval gives the second. The endpoint $x=-1$ is essential: looking only for a zero derivative would find the minimum and miss the maximum.

For comparison, the continuous function $g(x)=x$ on the open interval $(0,1)$ has infimum $0$ and supremum $1$, but neither is a value of $g$. Even a closed domain is insufficient without continuity: define $h(x)=x$ for $0\le x<1$, and $h(1)=0$. On $[0,1]$, its supremum is $1$, approached by $x_n=1-1/n$ for $n\ge 2$, but every function value is strictly less than $1$. This endpoint change makes $h$ discontinuous, preventing attainment. On the unbounded real line, the continuous function $x\mapsto x$ has neither a global minimum nor a global maximum. These examples show what can fail when a hypothesis is dropped; functions outside the hypotheses may still have extrema.

The extreme value theorem is an existence tool for optimization: first establish that a continuous objective function on a nonempty compact feasible set has a best value, then investigate where it occurs. It neither locates the optimizing point nor promises uniqueness or differentiability. For example, the absolute value function $|x|$ on $[-1,2]$ has its global minimum at zero although no derivative exists there; a constant function has every input as a minimizer. This role in optimization is explained in University of Toronto MAT237 notes, section 1.4: compactness and the extreme value theorem.

A short proof outline identifies the mechanism. The continuous image of a compact space is compact; a nonempty compact subset of the real line contains its infimum and supremum. For a closed interval, one can instead choose inputs whose outputs approach the supremum, extract a convergent subsequence by the Bolzano-Weierstrass theorem, and use continuity to make its limiting output equal that supremum. The closed interval retains the limiting input. Repeating the argument for the infimum supplies the minimum.

If two nonnegative finite-valued convex functions are defined on all of a finite-dimensional space, their infimal convolution is finite, convex and continuous everywhere. Convex perturbation duality then gives strong duality and dual attainment at every argument. It does not require or imply attainment of the primal decomposition infimum.

More precisely, let $f,g:{\mathbb{R}}^n\to [0,\infty )$ be finite everywhere and write $h=f\square g$. Nonnegativity supplies a lower bound, while the two trial splits $(x,0)$ and $(0,x)$ giveMixing nearly optimal splits proves convexity, and continuity of a convex function applies because the effective domain is the entire finite-dimensional vector space. These statements concern the optimized value, rather than existence of an optimal split. For the continuity theorem and dual attainment under the equality constraint $u+v=x$, see Bertsekas, Convex Optimization Theory: A Summary, Propositions 1.3.11 and 5.3.3(a). The equality has a feasible point in the whole space for every $x$, and the displayed bound makes its optimal value finite.

A finite value without any optimal split. On $\mathbb{R}$, take the convex functions $f(u)=e^u$ and $g(v)=0$. Both are nonnegative and finite everywhere, butfor every $x$. The splits $(u,v)=(-k,x+k)$ have costs $e^{-k}\to 0$, yet the exponential function is strictly positive at every finite $u$. No split attains the infimum. The value function is even constant and differentiable; regularity of the value function does not prevent optimizing splits from escaping to infinity. For example, $k=1,3,6$ gives costs approximately $0.367879$, $0.049787$ and $0.002479$.

A sufficient extra condition for attainment at a fixed $x$ is that $u\mapsto f(u)+g(x-u)$ be a coercive function, meaning it tends to $+\infty$ as $\Vert u\Vert \to \infty$. Under the finite-valued assumptions it is a continuous function, so a sufficiently large closed ball contains a global minimum by the extreme value theorem. If this cost is also a strictly convex function, the optimal split is unique. The quadratic example above satisfies both conditions; the exponential function example does not satisfy coercivity.

Nonnegativity is sufficient, not necessary: finite lower bounds $f\ge A$, $g\ge B$ would give $h\ge A+B$ by the same argument. Removing all lower-bound control can lose finiteness even for finite convex functions: $f(u)=u$ and $g(v)=-v$ give $f(u)+g(x-u)=2u-x$, hence $h(x)=-\infty$. Nor does finiteness alone guarantee differentiability: choosing $f=g=|\cdot |$ gives $h(x)=|x|$ by the triangle inequality and the split $(x,0)$.

For $n=1$, differentiability at $a$ means that the difference quotient has a finite two-sided limit of a function. For $n\ge 2$, require derivatives through order $n-1$ throughout some open neighbourhood of $a$, and require the final difference quotientto exist as a finite real number. The last derivative need not exist elsewhere or be continuous at $a$. In particular, existence of $f^{\prime }(a)$ by itself does not justify taking a second derivative: the function $f^{\prime }$ must first be defined in a neighbourhood of $a$.

The displayed implicit ordinary differential equation has a singular initial point at the origin. On a nonconstant regular branch, treating $x$ as a function of $y$ gives $dx/dy+x\cot y=2\cos y$. The integrating factor $\sin y$ yields the first integral of an ordinary differential equationOne can verify this identity without dividing by $\sin y$: its derivative along a solution is $\sin y-(2\sin y-x)\cos y{y}^{\prime }=0$.

The initial point forces $C=0$, so $\sin y(x-\sin y)=0$. Near the origin this gives the two differentiable solutions $y=0$ and $y=\arcsin x$, with initial slopes zero and one. A division by $\sin y$ would lose the first solution. A piecewise switch between the two at zero would fail differentiability because their one-sided slopes differ.

There is no conflict with the Picard-Lindelöf theorem: solving explicitly for $y^{\prime }$ requires division by $2\sin y-x$, which vanishes at the initial point. A regular implicit equation or a local uniqueness theorem must be checked before claiming a unique solution through a singular point.

At an interior point where the regression function is differentiable, a symmetric compactly supported regression kernel makes the first-order bias of an estimator vanish asymptotically. In regular fixed-design nonparametric regression with $h\to 0$ and $nh\to \infty$, the local constant estimator then has squared bias of an estimator $o(h^2)$ and variance $R(K)/(nh)+o((nh{)}^{-1})$ for a unit-integral regression kernel. The differentiability remainder is pointwise and need not hold uniformly over a function class.

At a local minimum of continuously differentiable functions, a suitable constraint qualification guarantees the existence of Lagrange multipliers satisfying the KKT conditions. One sufficient local regularity assumption is that the gradients of the equality optimization constraints and the active constraints are jointly linearly independent. This assumption concerns the description of the feasible set, not just the smoothness of the objective function.^[ref]

To see why a hypothesis is needed, minimize $f(x)=x$ subject to $g(x)=x^2\le 0$. The feasible set is the singleton $\{0\}$, so $x_{\ast }=0$ is a global minimum. Yet $f^{\prime }(0)+\lambda g^{\prime }(0)=1$ for every finite Lagrange multiplier $\lambda$. Thus this convex optimization problem has an attained optimum without a KKT multiplier. The zero gradient of the active constraint has hidden the actual restriction $x=0$ from its first-order equation. Rewriting it as the equality $h(x)=x=0$ yields the unrestricted Lagrange multiplier $\mu =-1$.

For convex optimization, Slater's condition requires strict inequalities and affine equalities at a point in the relative interior of the common domain of a function. A finite optimum value then gives strong duality and dual attainment. Primal attainment and differentiability yield KKT multipliers. Affine inequalities may be nonstrict in the refined version.^[ref]

Slater's condition is sufficient, and need not hold for an optimum to exist. The singular example above also shows that mere primal attainment is not a substitute for a constraint qualification.