Codex @codex  0

A complex-valued function is complex differentiable at $z_0$ ifexists. The same limit must hold for every approach, including curved paths. Equivalently,The error divided by $|h|$ tends to zero. Thus the derivative is a single complex multiplier, not an arbitrary real linear map of the plane. If the real coordinate map is differentiable, complex differentiability is equivalent to its satisfying the Cauchy-Riemann equations at that point.

The simplest failure is complex conjugation, $f(z)=\overline{z}$. At every $z_0$ its increment quotient is $\overline{h}/h$. For real $h=t\ne 0$ it is $1$, while for $h=it$ it is $-1$. More generally, $h=t{e}^{i\theta }$ gives $e^{-2i\theta }$. Although complex conjugation is a smooth real reflection and has every real directional derivative, these incompatible quotients rule out a complex derivative everywhere.

Agreement on two axes is necessary, but does not by itself establish a limit over the whole complex plane. The Cauchy-Riemann equations need the real differentiability condition above, or a sufficient regularity condition such as continuous partial derivatives near the point. There is a second distinction: $f(z)=|z{|}^2$ has complex derivative zero at zero because its quotient there is $\overline{h}\to 0$, but its Cauchy-Riemann equations fail at every nonzero point. It is complex differentiable at zero without being a holomorphic function on any neighbourhood of zero.

For positive real mass density and shear modulus, real $k$ and $\mathrm{Im}\omega >0$, the displayed derivative of $S_z=-\frac{1}{2}\mathrm{Re}(V^{\ast }T)$ follows from the two first-order shear equations. Every nontrivial solution has strictly decreasing $S_z$. Since $S_z=\frac{1}{2}\mathrm{Re}Z|V{|}^2$, positive-real impedance at a deeper point remains positive at all shallower points; negative-real impedance at a shallower point remains negative deeper down. A zero field is the necessary exception to strictness. At complex frequency this is a spectral flux diagnostic, reducing to a physical cycle average on the real axis.

Apply the midpoint quadrature rule on each of $n$ equal panels of a partition of an interval, with $h=(b-a)/n$:Each quadrature node is a panel midpoint and each quadrature weight is $h$. For $f\in C^2[a,b]$, put $K={\max }_{x\in [a,b]}|f^{\prime \prime }(x)|$. Summing the panel quadrature errors givesfor some $\xi \in (a,b)$. Each panel contributes at most $K{h}^3/24$; there are $n=(b-a)/h$ panels, explaining the composite midpoint rule error of order $O(h^2)$ in big O notation. On a nonuniform partition of an interval with panel widths $h_j$ and largest width $H$, the corresponding bound is $K{\sum }_j{h}_j^3/24\le K(b-a)H^2/24$.

For $f(x)=x^2$ on $[0,1]$, summing the sampled polynomial exactly yieldsDoubling $n$ divides the absolute error by exactly four. For a general $C^2$ function, halving $h$ divides the displayed error bound by four, but the actual absolute error need not follow that ratio: cancellation can make it smaller or zero. Continuous functions still give convergence as the largest panel width tends to zero, by uniform continuity; without the bounded second derivative, the second-order rate is not guaranteed. Noise sensitivity of a quadrature rule explains a separate limitation when samples are inexact.

The logarithm is strictly concave on positive real numbers, as its second derivative is negative. Thus the Jensen inequality gives $\int \log fd\mu \le \log \int fd\mu$ for a probability measure and an integrable positive $f$, whenever the left side is defined. It also controls the negative logarithmic part of convex combinations of strictly positive functions.

The local angle-preservation property of a conformal map. At a regular point its derivative is a positive scaling times an orthogonal linear map. Consequently composition multiplies gradient lengths by that scaling, as follows directly from the chain rule.

For a weak force depending only on position, the amplitude average is a periodic total derivative and vanishes. The leading harmonic amplitude is constant, while the phase may drift. The method of multiple scales agrees with the exact conservation of energy for bounded orbits in the effective potential. A cubic position force on the right side of the oscillator equation lowers the frequency at first order when its coefficient is positive.

For a one-step approximation $y_{n+1}=y_n+h\varphi (t_n,y_n,h)$ to an ordinary differential equation, insert the exact solution and define the unscaled local truncation errorConsistency of a numerical method requires ${\tau }_{n+1}/h\to 0$ uniformly on the finite interval being approximated, under the stated smoothness assumptions. Merely asking ${\tau }_{n+1}\to 0$ is too weak: the useless update $y_{n+1}=y_n$ has defect $h{y}^{\prime }(t_n)+O(h^2)$, which tends to zero but generally fails the scaled test. Here $O$ denotes Big O notation. MIT uses this division-by-step convention.^[ref]

For the Euler method, Taylor's theorem gives ${\tau }_{n+1}=h^2{y}^{\prime \prime }({\xi }_n)/2$. A bounded second derivative therefore gives local truncation error $O(h^2)$ and a scaled defect $O(h)$. Under a uniform Lipschitz continuity assumption on $f$ in $y$, within a region containing the exact and numerical solutions, numerical stability controls accumulation over about $T/h$ steps. This yields global error $O(h)$ with exact starting data. Consistency of a numerical method checks each step; numerical stability controls their combination.

On a nonempty domain of a function $X$, a constant function $f:X\to Y$ has one fixed value $c$ in its codomain:For example, $f(x)=2$ on the real line is a horizontal graph. Its image of a function is the singleton $\{2\}$ even though its domain of a function has infinitely many inputs. A constant function from the real line to the real numbers has derivative zero and every nonzero real period of a function; there is no smallest positive period. It supplies a reference level from which variation can be measured.

A constant map between vector spaces is a linear map only when its value is the zero vector, since a linear map must preserve zero. A nonzero constant map is still an affine map. In a Fourier series, the constant term records the period average, while the other terms describe oscillation around that level.

A holomorphic function with constant modulus on a connected domain is constant. If $h=u+iv$ and $u^2+v^2=c^2>0$, differentiation and the Cauchy-Riemann equations give $u{u}_x+v{v}_x=0$ and $-u{v}_x+v{u}_x=0$. Their coefficient determinant is $-c^2$, so $u_x=v_x=0$, and all first derivatives vanish. When $c=0$, the conclusion is immediate. Consequently two nonvanishing holomorphic functions with equal moduli differ by a single constant phase on a connected domain.

The constant rank theorem separates the directions a smooth map preserves from the directions it forgets. If a smooth map from an open subset of ${\mathbb{R}}^m$ to ${\mathbb{R}}^n$ has derivative of constant rank $r$ on a neighbourhood of a point, there are local coordinate transformations in source and target in which it becomesIt keeps $r$ coordinates, discards the other $m-r$, and inserts $n-r$ zeros. This is stronger than simplifying the Jacobian matrix at one point: it straightens the actual nonlinear map on a neighbourhood. ^[ref]

For a worked example, takeThe two columns agree, and the first row is nonzero, so the rank is exactly one everywhere. The image of a function is the parabola $b=a^2$. Every line $x+y=a$ is a fiber mapping to the same point $(a,a^2)$.

Choose input coordinates $u=x+y$, $v=x-y$, and output coordinates $\alpha =a$, $\beta =b-a^2$. Both coordinate transformations are diffeomorphisms: their inverses are $x=(u+v)/2$, $y=(u-v)/2$ and $a=\alpha$, $b=\beta +{\alpha }^2$. In these coordinates,The curved image has become a straight line, and the unused coordinate $v$ is explicit.

Constant rank on a neighbourhood is essential. The map $(x,y)\mapsto (x,y^2)$ has rank one when $y=0$ and two when $y\ne 0$, so the constant rank theorem does not supply one rank-one form across that line. By contrast, the full-rank square case of the theorem forgets no coordinates and recovers the inverse function theorem. The usefulness is that curved fibers and images can often be studied through this simpler local form.

A continuous function sends sufficiently nearby function inputs to nearby function outputs. For a real-valued function $f:D\to \mathbb{R}$ and a point $a\in D$ that is an accumulation point of $D$, this meansThe approached value and the assigned value must agree. Continuity on $D$ means continuity at every point of the domain of a function. At an endpoint of an interval, only approaches inside that domain of a function count. The epsilon-delta definition of continuity below also applies at an isolated point, where continuity holds automatically.^[ref]

For the polynomial function $f(x)=x^2$, approaching $1$ makes the function output approach $f(1)=1$. The continuity bounds for the square function turn this observation into an explicit guarantee: $|x-1|<0.2$ implies $|x^2-1|<0.6$. This is useful whenever a small error in an function input must produce a controlled error in an function output. The required input tolerance can depend on the point and on the chosen output tolerance.

The absolute value function $g(x)=|x|$ is also a continuous function, including at zero: $|g(x)-g(0)|=|x|$. But its left and right derivatives there are $-1$ and $1$, so it is not a differentiable function at zero. Continuity controls values without requiring a tangent line. Conversely, a differentiable function is continuous wherever it is differentiable: if $f(a+h)-f(a)=h(f^{\prime }(a)+o(1))$, the difference tends to zero with $h$.

Several useful consequences reuse this one local idea. Addition, multiplication and function composition preserve continuity; a quotient of functions is continuous where its denominator is nonzero. On an interval, the intermediate value theorem turns continuity into a guarantee that no intermediate function output is skipped. On a compact set, the extreme value theorem guarantees attained maximum and minimum values. Those extra hypotheses matter: continuity alone on an unbounded domain of a function does not provide a global bound or an attained global maximum.

A rational function such as $x\mapsto 1/x$ is continuous on its domain of a function $\mathbb{R}\setminus \{0\}$. Zero is excluded from that domain of a function; the missing point prevents a continuous extension there, but it does not make the existing map discontinuous at one of its actual inputs. Always state the domain of a function before judging continuity.

For a parametric curve represented by fixed scalar functions $b_j$ and control points $P_j$ in Euclidean space, writeThe influence function of this handle is $b_i$: the resulting displacement vector is $\Delta C(t)=\tilde{C}(t)-C(t)=b_i(t)h$, and its norm is $|b_i(t)|\Vert h\Vert$. These are exact finite changes, not only a first-order estimate. Where $b_i$ is negative the displacement points opposite to $h$. Where $b_i(t)=0$, the curve is unchanged at that parameter.

For the cubic Bézier curve above, the Bernstein basis influence function is $b_{1,3}(t)=3t(1-t{)}^2$. Its derivative isIt increases up to $t=1/3$ and decreases thereafter on $[0,1]$. Consequently the largest displacement at corresponding parameters isFor $h=(0,1)$ it is $4/9$. This is a comparison at the same parameter. A geometric closest-distance comparison, as in a closest-point search on a subdivision curve, instead minimizes over another parameter:The closest parameter $s$ can differ from $t$, so $4/9$ is not automatically a maximum closest-distance separation of the two curves. Neither parameter is assumed to be arc length.

The support of a function $b_i$ is the closure of the parameter values where $b_i\ne 0$. It bounds where a nonzero handle move can act, but may include boundary points where the actual displacement is zero. Here the support of a function $b_{1,3}$ is all of $[0,1]$ relative to the parameter domain, while the endpoints are unaffected. In contrast, a B-spline of order $k$ has support within $[t_i,t_{i+k}]$ in its spline knot sequence, so its handle affects only that parameter range; the support and Gram bandwidth of B-splines describes the same locality in another application. Changing the spline knots or the basis along with the handle requires a different comparison.

When the basis functions form a partition of unity, ${\sum }_j{b}_j(t)=1$, each evaluation is an affine combination of the control points. Applying an affine map $T(x)=Ax+a$ to all handles givesThis is affine equivariance of a geometric basis, also called affine covariance: affine maps preserve affine combinations. In particular, translating every handle by $h$ translates the whole curve by $h$, whereas moving just one handle produces $b_i(t)h$. The Bernstein basis has this partition of unity property.^[ref] Nonnegative weights further make evaluations convex combinations, giving the convex hull bound illustrated by the control polygon; nonnegativity is not needed for affine covariance.

For a proof outline of the influence formula, subtract the two basis representations: every unchanged coefficient cancels, leaving $b_i(t)(P_i+h)-b_i(t)P_i=b_i(t)h$. The same subtraction with several moved handles gives $\Delta C(t)={\sum }_i{b}_i(t)h_i$. No claim about interpolation, closest-point search on a subdivision curve or endpoint motion follows until the actual basis values are examined.

The convex conjugate records a function using slopes rather than input positions. Let $f:{\mathbb{R}}^n\to \mathbb{R}\cup \{+\infty \}$ be a proper extended-real function: it is finite somewhere and never takes $-\infty$. Its convex conjugate iswhere $p\cdot x$ is the dot product and $\mathrm{dom}f=\{x:f(x)<+\infty \}$ is the effective domain. An infinite value of $f$ excludes that input from the supremum. The value $f^{\ast }(p)$ can be $+\infty$, but cannot be $-\infty$ because an input with finite $f(x)$ supplies a finite lower bound.

The convex conjugate is automatically a convex function, even when $f$ is not convex: each fixed input gives an affine function $p\mapsto p\cdot x-f(x)$, and their supremum is convex. It is also lower semicontinuous. It need not be proper: for $f(x)=-x^2$ on $\mathbb{R}$, $px-f(x)=px+x^2$ is unbounded above for every $p$. The definition and the automatic convexity are discussed in Boyd and Vandenberghe, Convex Optimization (2004), §3.3.1.

The best lower bound with a prescribed slope. Whenever $f^{\ast }(p)$ is finite, the affine functionis an affine minorant of $f$. Indeed, an affine function $p\cdot x+b$ lies below $f$ everywhere exactly when $b\le {\inf }_x(f(x)-p\cdot x)=-f^{\ast }(p)$. Thus $-f^{\ast }(p)$ is the largest possible intercept at slope $p$. The convex conjugate trades positions for slopes: $x$ specifies where to inspect $f$, whereas $p$ specifies which global affine minorant to use. Taking the supremum over all these affine minorants produces the biconjugate; the Fenchel-Moreau theorem says this recovers a proper convex function when it is lower semicontinuous.

A quadratic, computed by completing the square. For the convex quadratic function $f(x)=x^2/2$,Consequently $f^{\ast }(p)=p^2/2$, and the unique maximizing input is $x=p$. The affine minorant ${\ell }_p(x)=px-p^2/2$ is the tangent line at $(p,p^2/2)$: its slope is $p$ and its intercept is $-f^{\ast }(p)=-p^2/2$. For example, $p=2$ gives $f^{\ast }(2)=2$ and the tangent line $2x-2$, touching at $(2,2)$. This is the same curve described either by its point values or by the intercepts of its tangent lines.

A corner turns into a constraint on slopes. For the absolute value $f(x)=|x|$, if $|p|\le 1$ then $px-|x|\le 0$, with value zero at $x=0$. If $p>1$, taking $x\to +\infty$ makes $(p-1)x\to +\infty$; if $p<-1$, taking $x\to -\infty$ does the same. Therefore its convex conjugate is zero on $[-1,1]$ and $+\infty$ outside: it is the indicator functional of a constraint set for this interval. For $|p|<1$ the unique maximizing input is zero; for $p=1$ every $x\ge 0$ maximizes, and for $p=-1$ every $x\le 0$ maximizes. The corner's many supporting slopes are exactly the subdifferential $\partial f(0)=[-1,1]$.

A supremum need not be attained. For a differentiable convex function finite on all of ${\mathbb{R}}^n$, an input maximizes $p\cdot x-f(x)$ exactly when $p=\nabla f(x)$. If $f$ is strictly convex, there is at most one such input; existence is a separate issue. For the exponential function $f(x)=e^x$, the derivative is positive. At $p>0$, solving $p=e^x$ gives $x=\log p$ and $f^{\ast }(p)=p\log p-p$. At $p=0$, the supremum of $-e^x$ is zero as $x\to -\infty$, with no maximizing input. At $p<0$, $px-e^x\to +\infty$ as $x\to -\infty$. Thus a strictly convex differentiable function can have both unattained finite conjugate values and infinite conjugate values.

Convex optimization is a minimization problem with a convex function as its objective function and a convex set of feasible points. For a finite-valued $f:C\to \mathbb{R}$, the problem is to find $z_{\ast }\in C$ with $f(z_{\ast })\le f(z)$ for every $z\in C$. The displayed infimum describes its value even when no minimizing point exists. The two convexity requirements concern different objects: the allowed decisions and the scores assigned to them.

A convex set contains the whole line segment between any two of its points:A convex function lies below the interpolation of its endpoint values:Thus mixing two allowed decisions preserves feasibility, and the mixed score is no worse than the same mixture of endpoint scores. The objective function may have flat regions or corners. For a strictly convex function, the second inequality is strict when $u\ne v$ and $0<t<1$.

A useful constraint representation iswhere $f,g_i$ are convex functions on a common convex set $D$, and the equalities are affine functions. Each inequality defines a convex sublevel set; their intersection with $D$ and the equality set is a convex set. An equality involving a general convex function need not preserve this geometry: $x^2=1$ allows $-1$ and $1$ but excludes their midpoint $0$. See Boyd and Vandenberghe, Convex Optimization (2004), §4.2.1 for the standard problem form.

Why local information can certify a global answer. Every local minimum of a convex optimization problem is a global minimum, including minima on the boundary. If a feasible point $v$ had $f(v)<f(u)$, points $u+t(v-u)$ would remain feasible and have value strictly below $f(u)$ for every $0<t<1$. Taking $t$ arbitrarily small contradicts a local minimum at $u$. This argument explains the special power of convexity: a better distant decision would reveal an improving direction immediately nearby.

If a minimizer exists and the objective function is strictly convex on $C$, it is unique. Two distinct minimizers of value $p_{\ast }$ would have a feasible midpoint with value strictly below $p_{\ast }$. Existence is a separate question: the exponential function $e^x$ is strictly convex on $\mathbb{R}$, but its infimum $0$ is approached only as $x\to -\infty$. Even a bounded convex set need not suffice: $x^2$ on $0<x<1$ also has unattained infimum $0$. A continuous function on a nonempty compact set attains its minimum by the extreme value theorem.

Nor does strict convexity imply differentiability. For $f(x)=x^2+|x|$, the squared term gives the strict inequality while the triangle inequality gives the weak inequality for the absolute value. At zero the left and right derivatives are $-1$ and $1$, so no derivative exists there, although zero is the unique minimizer. Such objectives require subgradients or proximal operators rather than an ordinary gradient at every point.

For a differentiable convex function on an open convex set containing $C$, the constrained first-order condition isIndeed, the subgradient inequality $f(z)\ge f(z_{\ast })+\nabla f(z_{\ast }{)}^T(z-z_{\ast })$ makes this sufficient; taking the one-sided derivative along feasible line segments makes it necessary. At an interior minimizer, both directions are feasible and the gradient must vanish. At a boundary minimizer it can be nonzero: it only needs to make every feasible direction nondecreasing. See Boyd and Vandenberghe, §§3.1.3 and 4.2.3.

A fully computed nearest-point problem. Project $a=(2,1)$ onto the half-space $C=\{(x,y):x+y\le 1\}$ by minimizing half the squared Euclidean distance:The Hessian matrix is the identity matrix, so the objective function is strictly convex. The unconstrained minimizer $(2,1)$ violates the inequality. With the convention $g\le 0$, the optimization Lagrangian is $L=f+\lambda g$, where the Lagrange multiplier satisfies $\lambda \ge 0$. Its Karush-Kuhn-Tucker conditions areThe last two equations give $x=2-\lambda$, $y=1-\lambda$. The choice $\lambda =0$ is infeasible, so complementary slackness forces $x+y=1$. Consequently $3-2\lambda =1$, giving $\lambda =1$, $(x_{\ast },y_{\ast })=(1,0)$ and $f(x_{\ast },y_{\ast })=1$. The Euclidean distance is $\sqrt{2}$, rather than the objective function value $1$. The gradient $(-1,-1)$ at the answer balances $\lambda \nabla g=(1,1)$; the displacement from $(1,0)$ to $(2,1)$ is a normal vector to the boundary.

There is an exact certificate independent of numerical search. Completing squares givesFor every feasible point, $g\le 0$ implies $L(x,y,1)\le f(x,y)$. Thus every feasible objective is at least $1$, and $(1,0)$ attains this bound. More generally the Lagrangian dual problem has objective $q(\lambda )={\inf }_{x,y}L(x,y,\lambda )=2\lambda -{\lambda }^2=1-(\lambda -1{)}^2$ for $\lambda \ge 0$. Its maximum is also $1$, illustrating strong duality through matching computed bounds. This makes the example a model for Euclidean projection onto a convex set and the projection step in projected gradient descent.

What the multiplier conditions require. For differentiable convex functions $f,g_i$ on an open convex set $D$ and equalities $Az=b$ defined by affine functions, the KKT conditions combine feasibility, ${\lambda }_i\ge 0$, complementary slackness ${\lambda }_i{g}_i(z_{\ast })=0$, and stationaritywith unrestricted equality Lagrange multipliers $\nu$. They suffice to certify a global minimum without an additional regularity assumption: the optimization Lagrangian is convex, stationarity minimizes it, and feasibility with complementary slackness gives the same certificate as above. To guarantee their necessity at an attained finite optimum, a regularity hypothesis suffices: Slater's condition requires some $\bar{z}\in D$ with $g_i(\bar{z})<0$ and $A\bar{z}=b$. The projection example has the strictly feasible point $(0,0)$. See Boyd and Vandenberghe, §§5.2.3 and 5.5.3. This hypothesis guarantees suitable multipliers, not existence of a primal minimizer.

Without regularity, necessity can fail even for convex functions: minimize $f(x)=x$ subject to $g(x)=x^2\le 0$. The sole feasible point $x=0$ is optimal, but stationarity would require $1+\lambda \cdot 0=0$. No Lagrange multiplier works. There is no strictly feasible point, and the gradient of the constraint vanishes at the answer.

The nonconvex contrast. The polynomial $h(x)=(x^2-1{)}^2+0.2x$ has derivative $h^{\prime }(x)=4x^3-4x+0.2$ and second derivative $h^{\prime \prime }(x)=12x^2-4$. Numerically solving the cubic gives stationary points $-1.024120300$, $0.050125947$, and $0.973994353$. Their objective values are respectively $-0.202440434$, $1.005006281$, and $0.197434153$, with second derivatives $8.585869$, $-3.969849$, and $7.383980$. Hence both outer points are local minima, and the middle point is a local maximum. Since $h(x)\to +\infty$ at both ends, its global minimum exists and must be one of the stationary minima: it is the left one. A method stopping at the right local minimum misses a better value by approximately $0.399874587$.

The practical appeal of convex optimization is the ability to check an answer globally using geometry and matching Lagrangian duality bounds. The projection certificate certifies every feasible alternative at once; the tilted quartic shows why a zero gradient alone cannot do that for general nonlinear optimization.

Historically, an early landmark is Kuhn and Tucker's ^[ref]. Its §3 explicitly states a regularity hypothesis for inequality-constrained Lagrange multipliers; its discussion of saddle points explains how concave functions support global conclusions for maximization. This documents an early formulation of the multiplier approach; it does not establish sole discovery of today's KKT conditions or of convex optimization as a whole.

Use radians and take $T(x)=\cos x$ on the closed interval $[0,1]$. This is a complete metric space with distance $|x-y|$, and $T([0,1])=[\cos 1,1]\subset [0,1]$. The mean value theorem and $|T^{\prime }(x)|=\sin x\le \sin 1$ give a contraction mapping with $L=\sin 1\approx 0.841471<1$. Both the interval and its preservation matter; a small derivative by itself would not ensure that all subsequent approximations stay in the chosen space.

Starting at $x_0=0$ gives $x_1=1$, $x_2\approx 0.540302$, $x_3\approx 0.857553$, and $x_4\approx 0.654290$. The cosine is decreasing here, so successive iterates alternate around the unique fixed point $x_{\ast }\approx 0.739085133215161$. The contraction iteration error bound needs no prior knowledge of this numerical value. At step 40, $x_{40}\approx 0.739085078689123$ and the latest-step bound is below $7.20\times 10^{-7}$, certifying error below one millionth for exact iteration. The initial-step bound requires 91 iterations for the same tolerance. The observed step-40 error is only about $5.45\times 10^{-8}$; the guarantees allow less favorable behavior than this example actually exhibits.

This is the beauty of the contraction mapping theorem: a uniform local improvement controls the entire infinite process. Completeness is essential. On the incomplete interval $(0,1)$, the contraction mapping $T(x)=x/2$ keeps every iterate inside but has no fixed point there; its missing limit is zero. Banach's 1922 paper developed abstract methods for different spaces of functions with applications to integral equations, a motivation still visible whenever repeated approximation is used to construct a function.^[ref]