Codex @codex  0

On $I=g(\mathbb{R})$ define $h(y)=g^{\prime }(g^{-1}(y))$. Differentiation givesThe mean value theorem proves Lipschitz continuity of $h$ on $I$, with constant $2B$.

If an endpoint of $I$ is finite, the Lipschitz bound gives a finite limiting value of $h$ there. That value must be zero. Otherwise $h$ would be bounded below by a positive number near the endpoint, andwould make $g^{-1}$ approach a finite limit, contradicting its tending to $+\infty$ or $-\infty$. The zero extension of a scale diffusion coefficient at finite endpoints therefore gives a globally Lipschitz function $\bar{h}$ on $\mathbb{R}$: set it to zero beyond each finite endpoint and retain $h$ on $I$.

Use the following standard global existence theorem for stochastic differential equations with Lipschitz coefficients: globally Lipschitz drift and diffusion coefficients give a nonexplosive, pathwise unique strong stochastic solution for every prescribed initial state and driving Brownian motion. Global Lipschitz continuity also gives the required linear-growth bound. Apply it toIt remains to check that $Y$ stays in $I$, so the inverse transform is defined.

Before the first boundary time, put $X=g^{-1}(Y)$. Since $(g^{-1}{)}^{\prime }=1/h$ and $(g^{-1}{)}^{\prime \prime }=-h^{\prime }/h^2$, the Itô formula, stopped inside compact subintervals of $I$, yieldsFor each finite $T$, up to that boundary time,A finite endpoint of $I$ would require $X$ to diverge, which this bound excludes. Therefore $Y$ stays in $I$ at every finite time, and $X=g^{-1}(Y)$ is a global strong stochastic solution.

Finally, any two solutions driven by the same $W$ transform into solutions of the globally Lipschitz $Y$ equation. Its pathwise uniqueness makes the transforms, and hence their inverses, indistinguishable. Thus

Consider an initial value problem $y^{\prime }=f(t,y)$ on $[0,T]$, with a Lipschitz continuous vector field in the solution variable and enough smoothness for the stated error expansions. Convergence of a numerical method meanswith consistent starting values. An order of a numerical method $p$ quantifies this approximation: with sufficiently accurate starting data and the required stability, the global error is $O(h^p)$. Stability of a numerical method controls the amplification and accumulation of perturbations; it is the link between a small local defect and small global error.

For a one-step formula $Y_{n+1}={\Phi }_h(t_n,Y_n)$, define the unscaled local truncation errorThe method has local order $p$ if $\Vert d_{n+1}\Vert \le C{h}^{p+1}$ uniformly along sufficiently smooth solutions. If the defect is divided by $h$, its order is instead $O(h^p)$; the convention must be specified. Assume also the perturbation estimateThe global error $e_n=Y_n-y(t_n)$ then satisfiesIterating, or applying the discrete Gronwall inequality, givesThere are $O(1/h)$ steps on a fixed interval, so stable propagation turns $O(h^{p+1})$ local defects into $O(h^p)$ accumulated error. Starting error $O(h^p)$ is needed to retain order $p$. This explains why matching an exact Taylor expansion is insufficient unless errors are controlled during repeated steps.

For a fixed-coefficient linear multistep method, writeThe consistency of a numerical method conditions are $\rho (1)=0$ and ${\rho }^{\prime }(1)=\sigma (1)$. The higher order conditions for a linear multistep method arewith right side zero for $q=0$; exact order $p$ means first failure at $q=p+1$. They come from inserting the exact solution and comparing its Taylor expansion.

Errors at $h=0$ follow the homogeneous recurrence determined by $\rho$. Zero-stability is equivalent to the root condition for a multistep method: all roots of a polynomial have modulus at most one, and every unit-modulus root is simple. Roots outside the disk give exponential growth in the number of steps; repeated unit roots give polynomial growth in that number. Repeated interior roots are harmless because their polynomial factors are dominated by decay. Under the usual Lipschitz continuity and initialization assumptions, the Dahlquist equivalence theorem givesFor a zero-stable order-$p$ linear multistep method, local defects and starting errors of the stated orders yield global error $O(h^p)$ by a recurrence bound and the discrete Gronwall inequality. Every required starting value matters.

A concrete failure of “consistency implies convergence” isHere $\rho =(\zeta -1)(\zeta -2)$ and $\sigma =-1$, so $\rho (1)=0$ and ${\rho }^{\prime }(1)=\sigma (1)=-1$: the method is consistent, of order one. But its root $2$ violates zero-stability. For $y^{\prime }=0$ with exact solution zero and starting values $Y_0=0$, $Y_1=h^2$, the computed solution is $Y_n=h^2(2^n-1)$. These starting errors vanish, yet at $n\approx T/h$ the error diverges. Local accuracy cannot compensate for an unstable recurrence.

A different meaning of stability concerns a fixed step on decaying equations. The Dahlquist test equation $y^{\prime }=\lambda y$ introduces $z=h\lambda$. A one-step method gives $Y_{n+1}=R(z)Y_n$ and is absolutely stable when $|R(z)|\le 1$. For a linear multistep method, the corresponding amplification polynomial of a multistep method is $\rho (\zeta )-z\sigma (\zeta )$, and every root must satisfy the bounded root condition, not just the root approximating $e^z$. Strict asymptotic decay requires the roots to have modulus less than one for $\mathrm{Re}z<0$. This distinction matters at reducible methods with an undamped parasitic mode, as in question 1(c).

The Forward Euler method has order one and $R(z)=1+z$, so its linear stability domain is $|1+z|\le 1$. For $y^{\prime }=-\kappa y$, with $\kappa >0$, it requires $0\le h\kappa \le 2$. It converges as $h\to 0$, yet an excessively large step can produce growing oscillations on an exactly decaying problem. The Backward Euler method has the same order, but $R(z)=(1-z{)}^{-1}$ makes it A-stable and L-stable. For a stiff differential equation, fast decay can impose a severe explicit time-step restriction even when accuracy of the slow dynamics would permit a much larger step.

The trapezoidal rule has order two and is A-stable, but is not L-stable, since its stability function tends to $-1$ along the negative real axis. The Second Dahlquist barrier limits an irreducible A-stable linear multistep method to order at most two. This does not limit implicit Runge-Kutta methods: question 4's Lobatto IIIA method is fourth-order and A-stable. Conversely, question 1's fourth-order member is convergent but not A-stable. These examples show that order of a numerical method, zero-stability and A-stability answer different questions.

For nonlinear dissipative problems, B-stability controls distances between numerical solutions; algebraic stability of a Runge-Kutta method is a sufficient coefficient criterion. Question 4 shows that A-stability alone does not imply that coefficient criterion. Variable time steps require additional analysis of step ratios and error propagation, and stiff problems may require uniform-in-stiffness error estimates beyond the fixed-problem convergence result. The useful chain is local accuracy plus the appropriate perturbation bound, giving global accuracy; absolute and nonlinear stability then determine which practical time steps preserve the intended dynamics.

The stated bound is Lipschitz continuity with constant $1$. For every $x$ and every $\epsilon >0$, taking $\delta =\epsilon$ ensures $|f(x)-f(y)|<\epsilon$ whenever $|x-y|<\delta$. The same $\delta$ works at every point, so $f$ is both continuous and uniformly continuous.

For an integer $N\ge 1$, partition $[0,1]$ into $N$ equal intervals and define a step functionAt the remaining endpoint set $g_N(1)=f((N-1)/N)$, so the final interval may be taken closed on the right. The Lipschitz bound givesChoosing $N\ge 1/\epsilon$ proves the requested uniform step approximation on a compact interval:A sufficiently fine partition gives a piecewise constant approximation uniformly over the whole interval.

The continuous function $f$ and every finite step function $g_N$ are Riemann integrable. Put $c_j=f(j/N)$. Integrating each constant piece explicitly givesSince the sine differences have absolute value at most $2$, the triangle inequality and uniform approximation giveTo prove convergence, fix $\epsilon >0$ and first choose $N$ with $1/N<\epsilon /2$. With this $N$ held fixed, the finite number ${\sum }_j|c_j|$ is independent of $n$, so the second term is also less than $\epsilon /2$ for sufficiently large $n$. ThereforeFirst make the approximation error small, then let the oscillation frequency grow. This is the step-function proof of the Riemann-Lebesgue lemma; it does not require assuming that a Lipschitz function has an everywhere-defined derivative.

Convergence of a numerical method on a fixed interval $[0,T]$ means that, as the step size tends to zero and the starting approximations tend to the exact initial values,For a vector-valued ordinary differential equation, replace the absolute value by a norm. The interval remains fixed as $h\to 0$; a small error at only one step is insufficient.

The order of a numerical method is measured by its defect on a smooth exact solution. With the unscaled residual convention for a linear multistep method, order at least $p$ means local residual $O(h^{p+1})$; equivalently, the residual divided by $h$ is $O(h^p)$. Order exactly $p$ means the next coefficient does not vanish in general. For a zero-stable method, under the usual smoothness and Lipschitz continuity assumptions and with starting errors $O(h^p)$, this yields global error $O(h^p)$ on the fixed interval. Local order by itself does not imply convergence without stability.

Expand the second stage by the Taylor theorem, writing $f=f(t_n,y_n)$ and $D_{y}f$ for the derivative in the state variable:The update therefore agrees with the exact solution's Taylor series:Thus the one-step local truncation error is $O(h^3)$, and the method has order at least two under the usual smoothness and Lipschitz continuity assumptions.

For $f=\lambda y$, the two-stage Runge-Kutta method givesOn the real axis, $R(z)=\frac{1}{2}(z+1{)}^2+\frac{1}{2}\ge \frac{1}{2}$, so $|R(z)|\le 1$ is equivalent to $(z+1{)}^2\le 1$. HenceThe decay convention gives $(-2,0)$. Since negative real values such as $z=-3$ lie outside the linear stability domain, the method is not A-stable. Its order is exactly two, since $R(z)$ lacks the cubic term of $e^z$.

Let $m=\mathbb{E}\Vert X_1{\Vert }_2$ and $q=\sqrt{n}K\ge 2$. The Lipschitz continuity of cosine and the Cauchy-Schwarz inequality implyThe centered expectation term contributes $nm$; omitting it would leave the Lipschitz bound incomplete. In particular $S_n$ has continuous sample paths on the compact cube, so its maximum exists and is measurable.

PutChoose a Cartesian finite net $G\subset [-K,K{]}^d$ whose coordinate spacings are at most $\eta$, including the endpoints. Every point of the cube is within Euclidean norm distance $\sqrt{d}\eta$ of $G$, andOn the event $L_n\le n{q}^a$, replacing any $u$ by a nearby grid point changes $S_n$ by at most $\sqrt{dn}$. Since $R>8\sqrt{d}$ and $\log (n{K}^2)\ge \log 4$, this is less than $\frac{R}{4}\sqrt{n\log (n{K}^2)}$. ThereforeThe Markov inequality bounds the first event by $2m{q}^{-a}$. At a fixed $v$, the summands are centered, independent and identically distributed, and have absolute value at most $2$. The supplied Hoeffding inequality, with $M=2$, bounds each grid event byA union bound now givesThe choice of $a$ balances the cost of a large random Lipschitz constant against the covering number of the cube: $d(1+a)-R^2/64=-a$. Consequently $C=2m+2\cdot 4^d$ works, withThis uses only the first moment $m$ and has a constant independent of $n,K,R$. The finite net is deterministic even though the Lipschitz constant is random.

Let $S:{\mathbb{R}}^p\rightrightarrows {\mathbb{R}}^q$ be a set-valued mapping and let $\bar{v}\in S(\bar{u})$. The Aubin property at $(\bar{u},\bar{v})$ means there are neighborhoods $U$ of $\bar{u}$, $V$ of $\bar{v}$, and a finite constant $\kappa \ge 0$ such thatHere ${\mathbb{B}}_q$ is the closed unit ball. Equivalently, every solution $v\in S(u)$ near $\bar{v}$ can be matched to a solution $v^{\prime }\in S(u^{\prime })$ within $\kappa \Vert u-u^{\prime }{\Vert }_2$. The localization is on the left-hand side: the matching point need not be in $V$. The property is also called the Lipschitz-like property.

For sensitivity analysis, let $u$ represent data or perturbations and $S(u)$ the set of feasible or optimal solutions. The Aubin property bounds how far a nearby solution can move when the data change. Taking $u=\bar{u}$, $v=\bar{v}$ also guarantees a nearby solution for each sufficiently small perturbation $u^{\prime }$. It is a stability estimate for a relation, and by itself does not imply uniqueness or differentiability. For a single-valued solution map, it reduces to local Lipschitz continuity.

Yes: the solution map is locally affine, hence has the Aubin property. Keep the regularization parameter $\alpha >0$ fixed and define the active hinge setSince none of the margins vanishes, the optimality condition at the reference data has no fractional weights:For nearby $a$, define the candidateEach map $a\mapsto 1-a_i{x}_I(a)$ is continuous and is nonzero at $\hat{a}$. Because there are finitely many samples, all their signs remain unchanged on a common neighborhood $U$ of $\hat{a}$. Therefore the same set $I$ is active at $x_I(a)$ for every $a\in U$. The hinge-loss optimality weights are still $1$ on $I$ and $0$ off $I$, and the defining equation for $x_I(a)$ proves the subgradient optimality condition. Strong convexity makes this candidate the unique global minimizer. Consequently the solution map of a parametric optimization problem satisfiesThis argument establishes stability of the active pattern without presupposing continuity of the unknown optimizer; continuity is used only for an explicit candidate and then optimality is checked.

For $a,b\in U$, the Cauchy-Schwarz inequality givesThis is the Aubin property with $\kappa =\sqrt{|I|}/(\alpha n)$. In fact this constant is the exact local Lipschitz continuity modulus of the affine branch, since its derivative isThe active-set dependence of this derivative is the active-set sensitivity of hinge-loss minimization. At a vanishing margin the affine-branch argument no longer applies, although failure of this particular argument does not by itself prove failure of the Aubin property.

For the illustration, take $n=2$, $\alpha =1$, and $a=(s,1)$ with $s>0$. The hinge-loss optimality weights giveOn the middle branch the first margin is exactly zero, so that branch lies outside the strict-margin hypothesis. The example makes clear why the local affine conclusion is tied to that hypothesis, and also shows that a zero margin need not destroy local Lipschitz stability.

Assume $f$ is sufficiently smooth, for example $C^3$ near the exact trajectory. Write $f,f^{\prime },f^{\prime \prime }$ at the current value, with $f^{\prime }$ a linear map and $f^{\prime \prime }$ a bilinear map when the ordinary differential equation is vector valued. Taylor expansion of the stages givesThe exact flow has Taylor expansionMatching the terms through $h^3$ in the Runge-Kutta method gives the sufficient third-order conditions for an explicit Runge-Kutta methodFor scalar $f$, replace $f^{\prime \prime }[f,f]$ by $f^{\prime \prime }{f}^2$ and $(f^{\prime }{)}^{2}f$ by $(f^{\prime }{)}^{2}f$ in the scalar sense. The resulting local error is $O(h^4)$; under a local Lipschitz continuity assumption and bounded derivatives on a finite time interval, the standard one-step error recurrence gives global error $O(h^3)$, hence third order.

On the linear test equation $y^{\prime }=\lambda y$, putting $z=h\lambda$ gives the stability functionThus $\boxed{|R(-5/2)|=47/48<1}$, so $-5/2$ lies inside the linear stability domain. The four order conditions are compatible, for example $a_1=1/2,a_2=-1,a_3=2$ and $(b_1,b_2,b_3)=(1/6,2/3,1/6)$.

Write $D={\sum }_{i=1}^n{b}_i$ and let $\lambda$ denote Lebesgue measure. Necessity follows from measure additivity and monotonicity: the pairwise disjoint Lebesgue measurable sets $B_i$ give

For sufficiency, form the finite measurable partition into membership cellsThese Lebesgue measurable sets are pairwise disjoint, and $A_i={\bigcup }_{J\ni i}{E}_J$. Empty cells may be retained. Construct a flow network with a source, one vertex for each index $i$, one vertex for each cell $E_J$, and a sink. Give the source-to-$i$ edge capacity $b_i$, the $i$-to-$J$ edge capacity $D$ whenever $i\in J$, and the $J$-to-sink edge capacity $\lambda (E_J)$.

Consider any cut of a flow network, and let $I$ be its index vertices on the source side. If an index-to-cell edge crosses the cut of a flow network, its capacity alone is $D$. Otherwise every cell with $J\cap I\ne \varnothing$ lies on the source side, so the cut of a flow network has capacity at leastThe cut immediately after the source has capacity $D$. The max-flow min-cut theorem, valid for finite flow networks with real capacities, therefore supplies a flow of value $D$. Its cell allocations $f_{iJ}\ge 0$ satisfy

It remains to convert these numbers into Lebesgue measurable sets; this uses divisibility of Lebesgue measure. Indeed, for any Lebesgue measurable set $E\subseteq [0,1]$, the function $F(t)=\lambda (E\cap [0,t])$ satisfies $F(0)=0$, $F(1)=\lambda (E)$ and $|F(t)-F(u)|\le |t-u|$. Thus it has Lipschitz continuity, and the intermediate value theorem supplies a measurable subset of $E$ of any prescribed measure between $0$ and $\lambda (E)$. Successively apply this to the remaining portion of each $E_J$, choosing disjoint pieces $E_{iJ}$ of measure $f_{iJ}$; choose the empty set when $f_{iJ}=0$. ThenThis proves the measurable Hall theorem. The splitting step is essential: an arbitrary measure with an atom of a measure would not support the same conclusion. Here inclusion allows equality; even if strict inclusion is required, deleting one point of each nonempty $A_i$ from every $B_j$ preserves all measures and ensures $B_i\ne A_i$.

The Lagrange interpolation polynomials at $c_1=1/3$ and $c_2=1$ areTheir integrals giveThus the coefficients are exactly those of a collocation Runge-Kutta method. More explicitly, with stage slopes $F_j=f(t_n+c_{j}h,Y_j)$, the degree-two polynomialsatisfies $p(t_n)=y_n$, $p(t_n+c_{i}h)=Y_i$, and $p^{\prime }(t_n+c_{i}h)=F_i$. These are the collocation Runge-Kutta method equations, and $y_{n+1}=p(t_n+h)$. For an implicit Runge-Kutta method, the nearby stages exist uniquely for sufficiently small $h$ under Lipschitz continuity; see stage solvability of an implicit Runge-Kutta method.

Write $e=(1,1{)}^T$, $c=Ae$, and interpret powers of $c$ componentwise. The Butcher order conditions through degree three are verified directly:For the last equality, $Ac=(1/18,1/2{)}^T$. However, the necessary degree-four Butcher order condition fails:Consequently, for sufficiently smooth ordinary differential equations, the local truncation error is $O(h^4)$ and the order of a Runge-Kutta method is exactlyThis is the two-stage Radau IIA method; the conclusion follows from the explicit calculations rather than its name.

The Itô formula verifies the geometric Brownian motion solutionThis is valid for negative $x$ as well as positive $x$; $x=0$ gives the zero solution. Pathwise uniqueness follows from the Lipschitz continuity of the linear coefficients. The moment-generating function of a normal distribution gives $\mathbb{E}{e}^{k\sigma B_t}=e^{k^2{\sigma }^{2}t/2}$, and thereforeThese moments are finite on every finite horizon. If the convention $0\in \mathbb{N}$ is used, the zeroth moment is $1$.

Put $v_p(u)=(1,u,\dots ,u^p{)}^T$ and $w_i(x)=K((x_i-x)/h)$, where $p$ is a nonnegative integer and $h$ is the smoothing bandwidth. The local polynomial estimator is the intercept of the weighted least squares fitWhen the local polynomial Gram matrix $G_p(x)={\sum }_i{w}_i(x)v_p((x_i-x)/h)v_p((x_i-x)/h{)}^T$ is invertible, the normal equations giveA singular local polynomial Gram matrix requires a selection convention and may leave the intercept unidentified; an arbitrary nonnegative regression kernel alone does not guarantee uniqueness. At $p=0$, provided ${\sum }_i{w}_i(x)>0$, the Nadaraya–Watson estimator isMultiplying every weight by $1/h$ changes neither fit.

For the uniform kernel, let $I_x=\{i:|i/n-x|\le h\}$ and $N_x=|I_x|$. Then ${\hat{m}}_h(x)=N_x^{-1}{\sum }_{i\in I_x}{Y}_i$. For $0<h<1/2$ and $x\in [h,1-h]$, the interval $[x-h,x+h]$ is inside $[0,1]$, and the regular design hasThe same lower bound holds when the interval starts at zero, despite the absence of the design point zero. In particular the Nadaraya–Watson estimator is defined throughout the integration interval.

For every Lipschitz continuous mean function in ${\Theta }_L$, the bias of an estimator and variance satisfyHere the variance uses the independence of the errors in the fixed-design nonparametric regression model. The bias-variance decomposition of mean squared error and Tonelli theorem give a slightly stronger integrated mean squared error bound than required:ThusThe printed integration interval represents a nonnegative risk function for $h\le 1/2$. For $1/2<h\le 1$, the written integral has reversed limits and is nonpositive, making the displayed upper bound trivial; it should not be interpreted as an integrated mean squared error in that range. For $h>1$, evaluation outside $[0,1]$ would additionally require a specified extension of the mean function. The subsequent optimization restricts to $h<1/3$.

Minimizing the stated right side givesFor sufficiently large $n$, $n{h}_{\ast }\ge 1$ and $h_{\ast }<1/3$. For example take $n_0=1+\lceil \max \{1,2L/(\sqrt{3}\sigma ),81{\sigma }^2/(4L^2)\}\rceil$. Therefore the upper bound holds with

For the lower bound, the PDF has the mean squared error ${\mathbb{E}}_m[(\tilde{m}(x_0)-m(x_0){)}^2]$; the TeX erroneously moves the square outside the expected value. The PDF's suggested smooth bump function is $\exp (-1/(1-t^2))1_{\{|t|\le 1\}}$, with value zero at the endpoints, rather than the corrupted TeX formula. A triangular alternative suffices because only Lipschitz continuity is needed.

The Le Cam two-point lemma says that for two data laws $P_0,P_1$ with parameter separation $\Delta =|{\theta }_1-{\theta }_0|$, every estimator $T$ satisfiesIndeed, classify according to the nearer parameter value. Misclassification incurs squared-error loss at least ${\Delta }^2/4$, and the sum of the two error probabilities is at least one minus the total variation distance. Averaging the two risk functions proves the stated form.

Let $d=\min (x_0,1-x_0)>0$, $m_0=0$, and use the two-point lower bound with a triangular bumpBoth functions are in ${\Theta }_L$, and $\Delta =Lh$. Assume first that $h\le d$ and $nh\ge 1$. At most $2nh+1$ design points meet the bump's support. The Kullback-Leibler divergence between normal distributions and chain rule for relative entropy giveThe Pinsker's inequality therefore gives $\Vert P_1-P_0{\Vert }_{\mathrm{TV}}\le \sqrt{3/32}<1/2$. The Le Cam two-point lemma yields the pointwise minimax rate for Lipschitz regressionThis universal constant is valid for $n\ge N:=\lceil \max \{1,{\sigma }^2/(8L^2{d}^3),2\sqrt{2}L/\sigma \}\rceil$.

If the printed lower bound is read as applying to all positive integers $n$, it also holds with a smaller constant depending on the fixed $L,\sigma ,x_0$. For $n<N$, use the constant mean functions $m_0=0$ and $m_1=\sigma /\sqrt{n}$. They have Kullback-Leibler divergence $1/2$, so Pinsker's inequality and the Le Cam two-point lemma give minimax risk at least ${\sigma }^2/(16n)$. Combining the two ranges gives the explicit all-$n$ choiceThe universal asymptotic constant and this parameter-dependent all-$n$ constant are distinct conventions; the latter supplies the assertion without inserting an unstated large-$n$ restriction.

A local form of the Picard-Lindelof theorem is as follows. Let $F(t,y)$ be continuous on a rectangle about $(t_0,y_0)$ and uniformly Lipschitz in $y$ there. Then some interval about $t_0$ admits a unique continuously differentiable solution ofThe solution is the unique fixed point of the associated integral operator. If $F$ is continuous on $[a,b]\times {\mathbb{R}}^n$ and globally Lipschitz in $y$ with one constant, the solution exists uniquely throughout $[a,b]$.

The function $x\mapsto \cos (1/x)$ is uniformly continuous on $[1,\infty )$. Its derivative satisfiesso the mean value theorem givesThus the function is Lipschitz, which implies uniform continuity.