The Volterra integration operator is injective and has dense range. Its range of the Volterra integration operator is , and its Moore–Penrose inverse of an operator is differentiation on this domain. Thus the error estimate for implicitly requires , in addition to the printed assumption. For an arbitrary function the same calculation bounds error against , but that function need not be in .
Put , and . The three stencils are forward, central finite differences, and backward on , and , respectively. All shifted evaluation points lie in . For , apply and change variables in each of the six evaluation terms. The resulting integration intervals are
Pair with , with , and the two middle intervals. Each pair has disjoint interiors, so almost every point is counted at most three times. This overlap multiplicity bound for piecewise difference operators proves
For the bias on the middle interval, write the central finite difference as an average of . The fundamental theorem of calculus gives
where . Extend by zero to the real line. Young's convolution inequality bounds the middle-interval error by . Combining its squared error with the supplied outer-interval estimate gives the central-difference noise-bias bound
The triangle inequality and the operator norm estimate now give
For , balancing noise and approximation bias minimizes this upper bound at
for sufficiently small that . Capping the parameter at supplies an admissible rule for all positive noise levels. More generally, and suffice; avoids needing or dividing by .
To prove convergence for every admissible datum, not just data, let with and extend by zero. Each stencil is a forward, centred or backward average of . Translation continuity in Lp makes each averaged difference from tend to zero in . Restricting those three bounds to their respective intervals proves . The noise-bias decomposition for linear regularization, with , finishes the convergent regularization of an inverse problem.
Take with the usual proper closed convex assumptions, so the Fenchel-Moreau theorem identifies its conjugate with the supplied . Put . The first update is . By Moreau decomposition, the second update is
Also , so the argument in the final update contains . Eliminating the auxiliary variable gives
This is the primal-dual hybrid gradient method with extrapolation parameter one and the primal update performed first. Its saddle point function is , corresponding to the primal objective .
For proper lower semicontinuous convex functions and a nonempty saddle-point set in these finite-dimensional spaces, the standard convergence of primal-dual hybrid gradient theorem gives the sufficient parameter condition
Here is the operator norm, or largest singular value. For , one possible choice is with ; for any positive steps satisfy the condition. The existence assumption is necessary: step sizes alone cannot guarantee convergence to a saddle point that does not exist.
For the divergence-form elliptic operator use the weak subsolution convention for nonnegative test functions. Set and test with , where . The Sobolev chain rule makes this test admissible; boundedness and the positive regularization control the derivative of its power. Ellipticity and the Cauchy-Schwarz inequality give, with and ,
Division when , and the trivial case , establish the power Caccioppoli inequality. Letting gives
For this limit uses the Fatou lemma; the gradient of a Sobolev function vanishes on a level set, so the weighted integrand is assigned zero on . The same regularization justifies differentiating in .
The PDF bounds individual entries by . Then : dimensional factors are included in the ellipticity constants when the first estimate is written as . With literal entrywise bounds, the explicit valid constant displayed here is . If bounds the operator norm instead, it is . This distinction does not alter any later conclusion, whose constants explicitly depend on .
For , put . Apply the Sobolev inequality to and use
For , , so the resulting constant is uniform in . Taking a cutoff equal to one on , supported in , with , yields
At a compactly supported cutoff can still be chosen with this bound by leaving an outer margin; alternatively pass to the limit from smaller radii.
For Moser iteration, set and . Repeated use of the preceding estimate bounds the successive norms by
This product is finite because
As the exponents tend to infinity, their norms on the Euclidean ball of radius tend to its essential supremum. Therefore
The power is independent of the exponents of Hölder continuity in earlier questions.
A pointed cone has no nonzero line through the origin: equivalently . If , its quadratic form vanishes on the nonnegative orthant. Testing the coordinate vectors gives . Testing then gives
Since is a symmetric matrix, every entry is zero. Thus .
The identity matrix gives an interior point. For a symmetric perturbation with operator norm , the Cauchy-Schwarz inequality gives
This open norm ball around lies even in the positive semidefinite cone, hence in . Consequently and the interior is nonempty.
More generally, the interior of the copositive cone consists exactly of strictly copositive matrices. Positivity on the compact nonnegative unit sphere has a positive minimum and persists under small perturbations; a zero there is destroyed by an arbitrarily small negative multiple of .
A Runge-Kutta method advances through stages
The Butcher tableau records , normally with . Linear absolute stability tests the amplification of a linear decay mode; nonlinear B-stability compares distances between two solutions of a dissipative vector field. Both describe propagation of perturbations, but they impose different conditions.
For the Dahlquist test equation, elimination of stages gives
The linear stability domain consists of for which the step is defined and . Poles or singular stage systems must be excluded, even if an output formula has a formal cancellation. For a scalar test equation, the proof is : the powers are bounded exactly when . For , the update is . If is a normal matrix, unitary diagonalization of a normal matrix makes the operator norm equal to the largest scalar amplification modulus. For a diagonalizable matrix, powers are bounded by the condition number of the eigenvector matrix times the maximal scalar power. These constants must remain uniform for mesh-dependent systems. A defective unit-modulus eigenvalue can instead produce a growing Jordan block; the scalar spectral test alone does not control a general matrix.
An A-stable method accepts the entire closed left half-plane. The Explicit Euler method has and a disk of stability , so it is not A-stable. No nonconstant polynomial stability function can be A-stable, since it is unbounded on the negative real axis; this excludes all consistent explicit Runge-Kutta methods. The Backward Euler method has , and when , proving A-stability. It is also L-stable, because for large left-half-plane .
The implicit midpoint rule and the trapezoidal rule have the same scalar stability function,
The identity proves A-stability. On the imaginary axis their amplification has unit modulus, and along the negative real axis it tends to . Thus neither is L-stable; very stiff decay can persist as alternating numerical values. In contrast, the two-stage Radau IIA method has , with the explicit nonnegative modulus-gap proof in question 1, and is L-stable. This illustrates why L-stability matters beyond A-stability for a stiff differential equation.
For nonlinear problems, assume a real or complex Euclidean inner product and the dissipativity hypothesis
for all relevant states at each common time. Exact solutions satisfy
A B-stable Runge-Kutta method reproduces this nonexpansion of distances whenever its stages exist. Put , , and . Then . Expansion of the output squared norm, followed by substituting in its linear terms, proves the Runge-Kutta contractivity identity
If and is a positive semidefinite matrix, the first sum is nonpositive by dissipativity, and the second is nonnegative: expand each coordinate of the stage vectors to express it as a sum of nonnegative quadratic forms. These are exactly the conditions for algebraic stability of a Runge-Kutta method. Hence algebraic stability implies B-stability. Applying B-stability to the linear dissipative problem , or its two-dimensional real form, also proves A-stability whenever stages are well defined.
The Backward Euler method has , hence ; the implicit midpoint rule has , , hence . Both are algebraically stable and B-stable. For the two-stage Radau IIA method,
a positive semidefinite matrix, while both weights are positive. It therefore combines third-order accuracy, B-stability, and L-stability. The distinction between stability and solvability is essential: for Lipschitz continuous , the stage map is a contraction when , so the contraction mapping theorem proves small-step unique solvability. The preceding algebraic stability proof is an estimate for existing stages; it is not a theorem asserting stage existence for arbitrary step sizes and domains.
Linear A-stability is insufficient for nonlinear B-stability. The trapezoidal rule fails B-stability, even though its scalar stability function is identical to that of the B-stable implicit midpoint rule. Its Runge-Kutta method coefficients are
Failure of this sufficient algebraic stability test alone would not prove failure of B-stability. An actual counterexample does: take and . Dissipativity follows from
The unique step map obeys , because the left-hand side is strictly increasing and onto. At , , and implicit differentiation gives
Nearby initial states expand in distance. This proves a true nonlinear failure rather than merely failure of a coefficient criterion.
Finally, stability of a numerical method on finite intervals connects these contractivity properties to accuracy. If a one-step map is nonexpansive and its exact one-step defect has norm at most , then the error satisfies , hence for . A Lipschitz step bound gives the same order using the discrete Gronwall inequality. For a stiff differential equation, the value of B-stability is that the propagation estimate need not grow with a large negative dissipative rate. The accuracy constant still needs appropriate smoothness and uniform derivative bounds; stability alone does not establish a mesh-uniform error order.
For a fully discrete linear partial differential equation evolution, write in a specified discrete norm. Finite-time stability of a numerical method means
with independent of the allowed meshes. For a multistep method, augment the state with its time-history values and include a stable starter. Bounds may grow as when the continuous problem has growth; demanding decay for all time would be a stronger assertion. The treatment of initial data, forcing, Dirichlet boundary conditions, and the norm is part of the hypothesis, not a detail supplied by an interior calculation.
For constant coefficients on an infinite or periodic uniform grid, Von Neumann stability analysis substitutes the Fourier mode . The discrete Fourier transform and the Parseval identity turn a uniform multiplier estimate into a discrete L2 norm estimate. In a one-step scalar scheme, proves contractivity, while gives finite-time stability. In a multilevel scheme, every root of a polynomial in its amplification polynomial of a multilevel finite difference scheme matters, as does uniform control of the associated companion matrix.
For the Forward Euler diffusion scheme with ,
Thus for all frequencies exactly when
Sufficiency follows since ; necessity follows from the highest-frequency mode on even periodic grids. This gives the usual parabolic mesh restriction . For the Backward Euler diffusion scheme,
so every is stable. The Crank--Nicolson method has
again of modulus at most one for all , but stiff modes approach rather than zero. These are PDE manifestations of the A-stability and L-stability distinctions for time integration.
For advection , assume and set . The upwind finite difference scheme is , with
It is contractive when . This also has a direct maximum-norm proof: each newest value is a convex combination of two old values. The resulting Courant–Friedrichs–Lewy condition expresses that the numerical domain of dependence covers the physical one. For negative , the upwind direction must be reversed. By contrast, Forward Euler method time stepping with the centered first finite difference has and for nonzero . Under fixed nonzero refinement it is unstable, since a fixed nontrivial mode grows geometrically over steps. This is not a claim of instability under every imaginable mesh coupling: instead bounds its spurious finite-time growth, since .
The leapfrog advection scheme illustrates a multilevel subtlety. Its amplification polynomial of a multilevel finite difference scheme is
For , both roots of a polynomial have unit modulus and separation at least . A uniformly conditioned eigenbasis of the companion matrix then proves power boundedness of a two-level Fourier scheme for arbitrary history data. At and a grid admitting , the roots coincide on the unit circle, and a Jordan block produces growth. Thus the endpoint fails the ordinary arbitrary-history root condition for a multistep method; checking only that both moduli equal one misses the instability. Restricting the starter or filtering the parasitic mode is an additional hypothesis.
The energy method handles variable coefficients and finite boundaries for which Fourier stability analysis may be unavailable. For the Backward Euler diffusion scheme with homogeneous Dirichlet boundary conditions, define . The discrete summation by parts identity is
Take the inner product of with . The elementary identity gives
This proves unconditional contractivity including the actual boundary treatment. More generally, a dissipative operator gives of operator norm at most one: with , dissipativity implies , and the Cauchy-Schwarz inequality gives . In finite dimensions this also proves invertibility. For the second-order backward differentiation formula, the BDF2 discrete energy identity used in question 4 controls both time levels and proves unconditional diffusion stability. A repeated root strictly inside the disk is harmless here; the energy argument supplies the uniform bound without a singular eigenvector formula.
A second technique is eigenvalue stability analysis of a finite difference method. Under the method of lines, a time integrator advances by . If is a normal matrix, the scalar linear stability domain criterion on all controls its powers exactly in the corresponding L2 norm. For the centered Dirichlet discrete Laplacian, its eigenvalues lie between and zero; intersecting this interval with a time integrator's stability interval determines its mesh restriction. A uniform bound on the diagonalization of a matrix is needed for nonnormal diagonalizable systems. Eigenvalues alone can be misleading. For example,
has both eigenvalues inside the disk for , but the upper-right entry of is . Taking , , this grows like . Hence stable scalar eigenvalues do not give mesh-uniform stability. Boundary closures can introduce precisely the extra growth that an interior Fourier symbol overlooks, so boundary stability of a finite-difference method must be checked separately.
For nonlinear spatial discretizations, a useful replacement for Fourier stability analysis is a convexity argument. If the Forward Euler method map is nonexpansive in a chosen norm for , the second-order strong stability preserving Runge-Kutta method
is also nonexpansive under the same restriction. Indeed, for two inputs, the triangle inequality gives . Its Taylor expansion is , proving order two. Such strong stability preserving Runge-Kutta methods transfer suitable forward-step bounds without relying on a linear spectral calculation.
Finally, consistency of a numerical method explains what stability buys. For a linear Hadamard well-posed problem, the Lax equivalence theorem equates convergence of a consistent discretization with its stability, under the stated approximation-space and norm hypotheses. Directly, if the error satisfies , iteration gives the discrete Duhamel principle
Thus vanishing normalized local truncation error and initial error yield convergence. With a nonlinear Lipschitz step estimate, the discrete Gronwall inequality plays the same role. A stable but inconsistent stencil, such as the literal defective formulas in questions 3 and 4 under their usual refinement, is not rescued by any amplification bound. The decisive checks are a mesh-uniform evolution bound, a valid boundary closure, and consistency with the actual PDE.
The continuous dual space consists of all bounded real linear functionals on , with operator norm . Let be Cauchy in this norm. For each , , so converges in ; define its limit to be .
Pointwise limits of the linear identities show that is linear. The sequence of norms is bounded, say by , so and . Given , choose so that for all . Holding fixed and taking the pointwise limit in gives for all . Thus , proving
Completeness of is not needed; completeness of the scalar field supplies the pointwise limit.
Let be the interior-grid matrix representing , with zero boundary error. It is real symmetric and has the stated sine eigenbasis. Put and ; the negative of its eigenvalue is
This increases in each variable on . With , the smallest absolute eigenvalue is at least . Hence its Euclidean norm operator norm satisfies
The corrected scheme has an unscaled residual whose individual entries are uniformly . There are entries and , so . Subtracting the exact nodal equations from the numerical equations gives (with sign depending on the residual convention), and therefore
Choose a constant slightly larger than the resulting bound to obtain the requested strict for all sufficiently small . Uniform smoothness is needed for an -independent constant; unspecified smoothness only in the interior would not by itself guarantee it.
For , , giving an upper bound for the operator norm. By Urysohn lemma, there is with . Then has uniform norm and . Thus every distinct pair has operator norm , and
The stated hypotheses also permit a singleton . Then there are no distinct pairs, so the infimum is under the extended-real convention, or undefined as a real number. This edge case cannot be assigned the value .
Let , so . By linearity, . The preceding operator norm estimate gives
Moreover, has factors on the data singular vectors, and is zero on . Every factor is in , hence
Thus, for any fixed , one can take and .
The first constant cannot generally be chosen independently of the iteration count. On the l2 sequence space, take , , and noise . At the solution perturbation has norm
This disproves a uniform-in- interpretation of the stated first estimate. For a closed operator range, boundedness of the Moore–Penrose inverse of an operator instead supplies the uniform constant .
Write the residual of an inverse problem as . The iteration can be rearranged as
Applying gives . Since is a positive operator, its resolvent operator has operator norm at most one. Hence
More explicitly, taking the inner product of the residual equation with gives
This directly proves the same monotonicity by the Cauchy-Schwarz inequality, including the case . An unresolvable data component in remains unchanged.
The relevant norm is the operator norm induced by the complex Euclidean norm, also called the matrix 2-norm:
The maximum exists because the unit sphere in finite dimensions is compact. It obeys and is the norm relevant to the specified perturbation energy. Other matrix norms do not in general give this gain.
The matrix exponential is the absolutely convergent power series
It converges in any finite-dimensional operator norm, since . Termwise differentiation gives , , so the solution is . Also , proving invertibility without requiring itself to be invertible.
The question's expansion in eigenvectors tacitly needs an eigenbasis: an invertible matrix alone need not be a diagonalizable matrix. The exponential solution and the singular-value arguments remain valid without that assumption.
Let with the supremum norm. By equivalence of norms in finite dimensions, there is such that whenever . Hence
so . This upgrades convergence on fixed polynomials to operator norm convergence on the fixed space.
For sufficiently large , , and a Neumann series gives
Thus the inverse Bernstein approximation on a fixed-degree polynomial space satisfies . By the extreme value theorem, strict positivity on the compact interval gives . Eventually , and consequently
Applying pointwise convergence to the varying polynomials without this finite-dimensional operator argument would not justify the conclusion.
Let be Cauchy in the operator norm on . For each ,
so is Cauchy in the Banach space . Define
Pointwise passage to the limit shows that is linear. A norm-Cauchy sequence is bounded, say , and hence
Thus is bounded. Finally, for fixed ,
The right-hand coefficient tends to zero with , so in operator norm. Therefore
This is the Banach space of bounded linear operators.
For finite-dimensional matrices , the Lie-Trotter product formula states
where convergence is in operator norm. In particular, for Hermitian matrices , set to obtain a product of unitary operators approximating .
The spectral norm is the operator norm induced by the Euclidean norm:
The Pauli operators and act on different qubits, and each is a unitary operator. Their product is therefore unitary and preserves the norm of every vector. Hence
Use the intended strong convexity bound . The original PDF omits on the right. The printed bound for every is impossible when , as shows. Equivalently, the intended bound is stated for unit vectors. The norm of is the operator norm induced by the Euclidean norm, or any matrix norm that bounds it.
Set
This function is , and its Hessian matrix satisfies
Part (i) therefore proves that is a convex function. The perturbation can use up the entire positive lower bound defining a strongly convex function; only convexity, rather than strict positivity of the Hessian matrix, is asserted.
The continuous dual space of a real normed vector space is
with the operator norm
Two normed spaces are isometrically isomorphic when there is a bijective linear map between them that preserves norms.
For , define
The series is absolutely convergent and
so and . For every , choose with and test on . This gives the reverse inequality, hence
Conversely, given , set . Then , so . For , its finite partial sums converge to in norm; continuity of therefore gives
The representing sequence is unique, and the construction is linear and norm preserving. Thus the duality of l1 and l infinity proves