Codex @codex  0

Real analysis studies limits, continuity, differentiation, integration, and convergence of sequences and functions taking values in the real numbers. Its central question is how finite approximations determine an exact answer: what does an arbitrarily small error mean, and which operations respect that error? A derivative is a limit of quotients, while a Riemann integral is obtained from increasingly fine Riemann sums. Both need conditions that make the answer independent of the approximations chosen.

Two organizing ideas are completeness of the real numbers, which ensures that suitable approximations have a destination, and uniform convergence, which controls approximation error over an entire domain. Pointwise convergence supplies separate guarantees at separate inputs; uniform convergence supplies one guarantee for all inputs. This distinction explains why replacing an approximating function by its limit can preserve continuity or integrals under appropriate hypotheses, yet fail without them.

For historical context, Cauchy's 1821 Cours d'analyse explicitly discusses convergence of series. Its Note VII treats double series and compares summing by rows and by columns under stated convergence assumptions: controlling an infinite process was already a concrete problem. This is historical context for questions about limits, rather than an attribution of the modern definitions below. ^[ref].

On $[0,1]$, use reduced representations of rational numbers with positive denominator. This reverses the nonzero values of the Thomae function, while leaving its values at irrational numbers zero. Every interval of positive length contains rational numbers of arbitrarily large reduced denominator: there are only finitely many fractions in $[0,1]$ with denominator at most a fixed $M$, whereas any open interval contains infinitely many rational numbers. Thus this function taking finite real number values is unbounded on every nondegenerate subinterval.

A power law $q(t)=t^{\rho }$ responds to every fixed rescaling $t\mapsto ut$ by the same factor $u^{\rho }$. Regular variation keeps this scaling behaviour in the limit $t\to \infty$, while allowing departures from an exact power law. It is useful when the scaling exponent is stable but a smaller correction continues changing. The correction can even grow without bound.

An eventually positive measurable function $q$ has regular variation at infinity with index $\rho \in \mathbb{R}$ ifThe argument $ut$ is in the domain of a function for sufficiently large $t$. Each $u$ is held fixed as the limit is taken; allowing $u$ to change with $t$ is a different assertion. This is the scalar definition recalled by ^[ref]. The index is unique: taking $u=2$ determines $2^{\rho }$.

For example, on $t>1$ the function $q(t)=t^2\log t$ hasThus $q$ has regular variation of index two. Yet $q(t)/t^2=\log t\to \infty$: regular variation does not require asymptotic equivalence to a constant times a power law. Doubling $t$ gives $q(2t)/q(t)=4(1+\log 2/\log t)$. At $t=100,10^4,10^6$ this is approximately $4.60206,4.30103,4.20069$, respectively. The ratio approaches four slowly, even though the exponent is exactly two in the limiting scaling law.

The slowly varying function isolates this correction. Set $\ell (t)=q(t)/t^{\rho }$. ThenConversely, multiplying a slowly varying function by $t^{\rho }$ gives regular variation of index $\rho$. Hence the decomposition $q(t)=t^{\rho }\ell (t)$ is exact on its eventual domain of a function; it is the scaling relation, rather than the decomposition itself, that is asymptotic.

A Pareto distribution gives a decreasing example with a direct interpretation. For $x\ge 1$, take the tail probability $\mathbb{P}(X>x)=x^{-2}$. Its regular variation index is $-2$. Doubling a threshold divides its tail probability by four, andFor a tail probability with regular variation of index $-\alpha$, the same conditional probability tends to $2^{-\alpha }$. An exponential distribution behaves differently: its corresponding conditional probability is $e^{-\lambda t}\to 0$. This comparison explains why scaling limits are useful in extreme value theory; it does not establish that an observed dataset follows a particular tail model. The role of regular variation in heavy-tailed distributions is discussed in Lindskog, Resnick and Roy, introduction, p. 1.

An exponential function $q(t)=e^t$ is not regularly varying with any finite index: at $u=2$, $q(2t)/q(t)=e^t\to \infty$, rather than $2^{\rho }$. Boundedness alone is also insufficient, as the oscillating counterexample in slowly varying function shows. Checking a few rescaling ratios at finite arguments can illustrate a proposed law, but cannot prove its limiting definition.

Jovan Karamata developed this theory in his 1930 paper and returned to its fundamental results in 1933. The opening of his 1933 paper identifies the earlier work; pp. 57–58 connect the rescaling limit with a power law and a slowly varying factor. See J. Karamata, Sur un mode de croissance régulière. Théorèmes fondamentaux, Bulletin de la Société Mathématique de France 61 (1933), 55–62. This gives a precise language for “almost a power” without discarding corrections that remain large.

The residue of a function $f$ at an isolated singularity $a$ is its Laurent coefficient of degree $-1$. More explicitly, if $f$ is holomorphic for $0<|z-a|<\rho$, write its Laurent series asThis one Laurent coefficient determines the contour integral around a sufficiently small counterclockwise circle about $a$. A complicated local Laurent series can therefore leave a remarkably simple answer after one complete trip around the point.

For a simple pole, multiplying by $z-a$ removes the pole, leaving a holomorphic extension whose value at $a$ is the residue:There is a useful shortcut for a quotient $f=g/h$. If $g$ and $h$ are holomorphic near $a$, $h$ has a simple zero there, and $g(a)\ne 0$, then $f$ has a simple pole andThe first-order Taylor series of $h$ explains the denominator: dividing $h(z)$ by $z-a$ leaves its derivative at $a$ in the limit. This computes a residue without finding the full Laurent series, which is especially useful when the numerator or denominator is complicated. ^[ref]

For example, the rational function $f(z)=1/(z^2+1)$ has simple poles at $i$ and $-i$. Factoring the denominator makes both residues immediately visible:The signs matter: these residues cancel when both poles are enclosed, but either one contributes when enclosed alone. The residue theorem below turns this distinction into different contour integrals of the same function.

A residue is not the same thing as a pole, nor does it measure the order of a pole. The function $1/z^2$ has a double pole at zero but residue zero. Conversely,has an essential singularity at zero and residue one. Infinitely many negative powers cause no difficulty in reading off that particular Laurent coefficient. At a removable singularity the residue is zero. ^[ref]

Why does only this Laurent coefficient survive? Parametrize a counterclockwise circle by $z=a+r{e}^{it}$, $0\le t\le 2\pi$, with $0<r<\rho$. For every integer $n$,When $n\ne -1$, the complex exponential completes a whole number of turns and its contributions cancel; when $n=-1$, it is constant. Uniform convergence of the Laurent series on this circle justifies termwise integration, giving $\oint f(z)dz=2\pi i\mathrm{Res}(f,a)$. Thus the residue isolates exactly the local information that a closed contour integral detects. ^[ref]

This selection rule explains why a very large value near an isolated singularity need not produce a large contour integral. The double pole $1/z^2$ grows faster near zero than $1/z$, yet its contributions around a circle cancel completely. What matters is the arrangement of complex values along the path, including the differential $dz$, rather than their moduli alone. A residue records that surviving contribution, not the size of the function near the point.

Let $\Gamma$ be a piecewise smooth contour forming a simple closed curve, positively oriented: travel counterclockwise, with the interior on the left. Suppose $f$ is holomorphic on an open set containing $\Gamma$ and its interior, except at finitely many isolated singularities $a_1,\dots ,a_m$ strictly inside. ThenNo isolated singularity may lie on $\Gamma$: the ordinary contour integral must be defined there. Reversing the direction of the contour reverses the sign of the answer. The theorem applies to essential singularities as well as poles. ^[ref]

Return to $f(z)=1/(z^2+1)$. A counterclockwise circle $|z|=2$ encloses both simple poles, soYet $f$ is nonzero everywhere it is defined. A zero contour integral does not imply a zero function, or even absence of poles: the enclosed residues can cancel. By contrast, the counterclockwise circle $|z-i|=1/2$ encloses only $i$, givingThe picture compares these two selections. Changing the shape of a contour while keeping the same enclosed poles leaves the contour integral unchanged, provided the contour deformation stays where $f$ is holomorphic.

This is the practical appeal of the residue theorem: local Laurent coefficients determine a global contour integral. For a rational function, a few simple pole computations can replace a long parametrized calculation. It also makes real improper integrals accessible by treating the real line as part of a closed contour in the complex plane.

To compute $I={\int }_{-\infty }^{\infty }dx/(1+x^2)$, take $R>1$ and travel along the real line from $-R$ to $R$, then return along the upper semicircle $A_R$, parametrized by $z=R{e}^{it}$ for $0\le t\le \pi$. This contour is positively oriented and encloses the simple pole $i$ only; $-i$ is outside. HenceThe added arc must be controlled before taking a limit. On $A_R$, the reverse triangle inequality gives $|z^2+1|\ge R^2-1$, and the arc length is $\pi R$. The estimation lemma therefore yieldsThe denominator grows quadratically while the arc length grows only linearly. This quantitative comparison, rather than a sketch alone, explains why closing the contour does not alter the limiting real improper integral. It even supplies an error bound: the difference between $\pi$ and the finite real integral is at most $\pi R/(R^2-1)$.

For a numerical check, $R=10$ gives the finite real integral $2\arctan (10)\approx 2.94226$, leaving about $0.19934$ to reach $\pi$. The arc bound is $10\pi /99\approx 0.31733$, comfortably larger than that error. The graph shows both quantities approaching zero. The exact tail here can be checked with the antiderivative $\arctan x$; the residue theorem method remains useful for improper integrals where a convenient real antiderivative is unavailable.

Finally, both one-sided improper integrals converge absolutely: for $|x|\ge 1$, $0<1/(1+x^2)\le 1/x^2$, whose tails are integrable. Thus the symmetric limit computes the ordinary improper integral, not merely a Cauchy principal value, andClosing below instead would enclose the simple pole $-i$. With the same left-to-right traversal of the real line, that contour is clockwise, so its contour integral is $-2\pi i\mathrm{Res}(f,-i)=\pi$ again. This provides a useful sign check: the opposite residue is accompanied by the opposite direction around the enclosed region.

The same strategy extends to many other rational functions: choose a contour, identify its enclosed isolated singularities, compute their residues, and bound the added pieces. The choice of contour and the bound are as much part of the calculation as the residues. ^[ref]

For a function $f:X\to Y$ and a subset $A\subseteq X$, its restriction $f{|}_A$ keeps the same values but allows only inputs in $A$. Restricting the domain of a function can remove collisions and permit an inverse function, as the nonnegative quadratic function example shows. This changes the allowed inputs; it is different from keeping the inputs and changing the codomain.

For a reversible Markov chain with stationary distribution $\pi$ and $\nu \ll \pi$, detailed balance givesTo prove this, integrate either side against a bounded test function and exchange the two endpoints using detailed balance. If $\pi =h/Z$, the output probability density function is $q_t(x)=h(x){\mathbb{E}}_{B\sim K^t(x,\cdot )}[\nu (B)/h(B)]$. A reverse simulation can therefore estimate this density ratio without evaluating a transition probability density function.

Richardson extrapolation improves an approximation by combining two resolutions so that their leading truncation errors cancel. The useful surprise is that the size of that leading error need not be known: its scaling under refinement is enough. This lets an existing finite difference method or quadrature rule produce a more accurate answer from two runs, without first deriving a new method from scratch.

Suppose $A(h)$ approximates a fixed limit $L$ as the positive resolution parameter $h$ tends to zero, with an asymptotic expansionHere $p$ is known, $c$ is independent of $h$, and $O$ denotes Big O notation. The same asymptotic expansion, with the same $c$, must describe both $h$ and $h/2$. Nested grids make it natural to reuse samples, although nesting itself does not establish this error model. Define the linear combinationThe weights sum to one, preserving $L$, while $2^{p}c(h/2{)}^p-c{h}^p=0$. Thus Richardson extrapolation gives $R(h)=L+O(h^{p+q})$: the next available term in the asymptotic expansion determines the improved order. Justin Solomon's numerical analysis text develops this cancellation for finite differences and applies it to numerical integration.^[ref]

Geometrically, regard the approximation as a function of $t=h^p$. Its first two terms are a straight line $L+ct$. The two computed points lie at positive $t$; extending the line through them to $t=0$ gives $R(h)$. This is why the operation is extrapolation: it estimates a value outside the sampled parameter interval. The remaining curvature corresponds to the uncancelled terms of the asymptotic expansion.

Worked example: a centered derivative of sine. For the smooth function $f(x)=\sin x$, the exact derivative at $x=0$ is $L=\cos 0=1$. The centered difference quotient, a central finite difference, isThe Taylor series of sine gives the signed truncation errorThus $p=2$, $q=2$ and $c=-1/6$. Halving $h$ divides the leading truncation error by four. Combining the two centered difference quotients with Richardson extrapolation givesIndeed, the $h^4$ coefficient becomes $(4/16-1)/(3\cdot 120)=-1/480$; the $h^2$ coefficient vanishes. This is the even-power case of Richardson extrapolation discussed in Persson's authored lecture notes.^[ref]

For $h=0.4$ and $h/2=0.2$, the actual computations areThe absolute error drops from about $2.6454\times 10^{-2}$ for $A(0.4)$ to $5.3080\times 10^{-5}$ for $R(0.4)$, an improvement by roughly a factor of 498. Refining again gives $R(0.2)\approx 0.999996670632607$, whose absolute error is about $3.3294\times 10^{-6}$. The refined truncation errors decrease by nearly four for $A$ and nearly sixteen for $R$, reflecting their second and fourth orders.

The logarithmic scale on each axis turns power laws into straight lines. The dashed reference curves are $h^2/6$ and $h^4/480$, the magnitudes of the leading truncation errors. The plotted errors are computed with 70-digit decimal arithmetic, rather than clipped to a display floor; this isolates the asymptotic expansion from roundoff error.

Worked example: integrating an exponential. Approximate the definite integral of the exponential function on $[0,1]$:For numerical integration, the composite trapezoidal rule adds the areas of trapezoids between successive samples; this is the quadrature use of the endpoint average that also occurs in the ODE trapezoidal rule. On $n$ equal intervals, with $h=1/n$, defineSumming the geometric series gives $T_n=(e-1)\frac{h}{2}\frac{e^h+1}{e^h-1}$. Expanding the exponential function near zero yieldsThe leading truncation error again has $p=2$, $q=2$. On the uniform nested grids with $n$ and $2n$ intervals, Richardson extrapolation therefore givesThis linear combination is exactly composite Simpson's rule on the finer grid, whose spacing is $h/2$. On each coarse panel, an old endpoint receives weight $h/6$ and the new midpoint weight $2h/3$; shared old endpoints receive weight $h/3$. These are the usual Simpson's rule weights $1,4,2,\dots ,4,1$ multiplied by $h/6$. The equivalence is explained directly in LeMesurier's university notes.^[ref]

For two coarse panels and four fine panels,The absolute errors are about $0.0356493$, $0.00894008$ and $0.0000370135$, respectively. Reusing the fine-grid samples makes Richardson extrapolation especially attractive here: the resulting Simpson's rule needs no samples beyond those already used by $T_{2n}$.

When the improvement is justified. Richardson extrapolation requires the stated asymptotic expansion and the correct positive $p$. If the actual leading exponent is $r\ne p$, its coefficient after combination is $c(2^{p-r}-1)/(2^p-1)$, so the leading truncation error generally remains. Merely knowing $A(h)-L=O(h^p)$ does not supply a common coefficient $c$ or a higher-order remainder. A discontinuity of a function, a singular endpoint, or unrelated changes between grids can invalidate the assumed asymptotic expansion; refinement alone does not guarantee improvement for every numerical problem.

Roundoff error and errors in the sampled values survive the cancellation of truncation error. If the coarse and fine values have absolute perturbations bounded by $\epsilon$, the perturbation in $R(h)$ is bounded by $(2^p+1)\epsilon /(2^p-1)$; for $p=2$ this is $5\epsilon /3$. In floating-point arithmetic, a centered difference quotient can also amplify sample errors through division by $h$, so an indefinitely smaller $h$ need not be more accurate. Persson's notes describe this balance between truncation error and roundoff error.^[ref]

The same cancellation idea appears in step-doubling error estimation. For an order-$p$ one-step method for an ordinary differential equation, two half steps each contribute local truncation error of order $h^{p+1}$, so their accumulated leading local error has ratio $2^{-p}$ to one full step. This explains the denominator $2^p-1$ there; the appropriate exponent follows from that local error model, rather than automatically from a generic use of Richardson extrapolation.

A right inverse of a function $f:X\to Y$ is a function $g:Y\to X$ satisfying $f\circ g={\mathrm{id}}_Y$. It forces $f$ to be a surjective function. A surjective linear map onto a finite-dimensional vector space has a linear right inverse: lift a basis of the target and extend by linearity.

For finite sets, the number of ordered choices of one element from each is the product of the individual numbers of choices. More generally, if stage $j$ has exactly $n_j$ possible choices after every permitted history of earlier stages, the total number of full choice sequences is ${\prod }_j{n}_j$, by mathematical induction on the stages. This counts $b^a$ functions between sets of sizes $a,b$, and counts the injective functions by the falling factorial $b(b-1)\cdots (b-a+1)$ when $a\le b$.

With five people, choose a captain in five ways and a different deputy in four ways: $5\cdot 4=20$ ordered assignments. The second choice depends on the first because the captain is excluded, but its number of permitted options is always four. The rule of product therefore applies without any assumption about independent events. To count two-person teams instead, each subset has two possible role assignments, so division by $2!=2$ gives ten teams. This division works because every team was counted equally often.

The Runge phenomenon is the failure of increasing-degree polynomial interpolation at equally spaced interpolation nodes to converge to some smooth target functions, with large oscillations near the ends of the interval. It is a property of the approximation problem, and can occur even with exact arithmetic. A numerically stable evaluation method cannot remove it. Endpoint clustering, as in Chebyshev-Lobatto nodes, changes the approximation problem and can greatly improve it.^[ref]

Take $f(x)=1/(1+25x^2)$ on $[-1,1]$ and polynomial degree at most $10$. Use either $x_k=-1+k/5$ or $x_k=\cos (k\pi /10)$ for $k=0,\dots ,10$. Each interpolation polynomial passes through all eleven of its own samples, yet the first has severe endpoint oscillations and the second stays much closer to the target function. Evaluating both by the barycentric interpolation formula on $20,001$ equally spaced points, including the endpoints, gives sampled maximum absolute errors of approximately $1.915659$ and $0.132197$, respectively: about a $14.49$-fold reduction. These are maxima over the stated grid, not certified maxima over the continuous interval. The generator separately checks exact reproduction of all stored nodal values.

The polynomial interpolation error formula explains why interpolation nodes enter the analysis. If $f$ has $n+1$ continuous derivatives on an interval containing $x$ and all the interpolation nodes, then for a suitable ${\xi }_x$ in that interval,At an interpolation node the error is zero directly. Away from the interpolation nodes, both the product and the high-order derivative matter. A short proof applies Rolle's theorem repeatedly to $f(t)-p_n(t)-[f(x)-p_n(x)]{\prod }_i(t-x_i)/{\prod }_i(x-x_i)$, which vanishes at $n+2$ distinct points.^[ref]

The successful comparison here does not assert that Chebyshev-Lobatto nodes solve every approximation problem for every function. The derivatives, the polynomial degree, and measurement errors still matter. The Lebesgue constant of interpolation measures worst-case amplification of nodal errors, while the polynomial reproduction error bound relates interpolation to the best uniform approximation available at that polynomial degree.

For a stochastic process $X_t$ and a fixed sample space outcome $\omega$, its sample path is the function $t\mapsto X_t(\omega )$. One row of observed fair bits, one trajectory of a random walk, or one staircase of Poisson process arrival counts is a sample path. Fixing $t$ instead gives a random variable ranging across possible histories. The same two-variable object $X_t(\omega )$ therefore describes both observations across outcomes and evolution within an outcome.

Statements about continuous, càdlàg or Hölder continuous function sample paths concern these functions simultaneously in time. They differ from assertions made at each fixed time separately. For example, take $U$ with a uniform distribution on $[0,1]$ and set $X_t=1_{\{t=U\}}$. For every fixed $t$, $X_t=0$ almost surely, but every sample path has a single spike. Thus this process is a modification of a stochastic process that is identically zero, while the two are not indistinguishable stochastic processes. Continuity of both sets of sample paths rules out this discrepancy: equality at a countable dense set then implies equality everywhere. This is why path regularity is an additional part of constructing a continuous-time model.

Let $f$ have continuous second partial derivatives near an interior critical point $x_{\ast }$ of an open subset of ${\mathbb{R}}^n$, with $\nabla f(x_{\ast })=0$. The Hessian matrix gives the following sufficient conditions:

A positive semidefinite or negative semidefinite Hessian matrix with a zero eigenvalue does not settle the classification. The higher-order test for separated extrema gives two quartic examples with exactly the same zero Hessian. These conditions apply to an interior critical point; a boundary optimum or a constrained optimization problem requires attention to the allowed directions. ^[ref]

The proof outline is the Taylor expansionFor a positive-definite matrix, the smallest eigenvalue supplies a positive bound proportional to $\Vert h{\Vert }^2$, which dominates the remainder for small enough $h$. Negate the function for the local maximum case. For opposite-sign eigenvalues, choose the two corresponding eigenvectors: the leading terms have opposite signs along their lines.

There are also necessary conditions. At an interior local minimum, the Hessian matrix is positive semidefinite; at an interior local maximum, it is negative semidefinite. Restriction to each straight line reduces this to the one-variable second derivative condition. Since the Laplacian is the trace of the Hessian, it is nonnegative at such a local minimum and nonpositive at such a local maximum. The sign of the Laplacian alone is insufficient: $x^2-2y^2$ has negative Laplacian but a saddle point of a scalar function at the origin.

The Haar wavelet provides an explicit one-vanishing moment comparison:It cancels a constant polynomial, but not a nonconstant affine function. To obtain two vanishing moments by an equally transparent construction, take three adjacent unit intervals with heights proportional to $1,-2,1$:The L2 norm normalization follows from $1^2+(-2{)}^2+1^2=6$. Direct integration givesThus $\eta$ has exactly two vanishing moments. In fact its coefficient is a finite difference of adjacent local integrals:The weights are those of the second-order central difference. The first moment balances the centres of the three intervals; it is not enough merely to balance their signed areas.

This $\eta$ is a normalized analyzing function, with no orthonormal basis assertion. Indeed, its unit translate has nonzero L2 inner product:Hence these translates already fail orthogonality. The Daubechies wavelet construction imposes additional refinement and orthogonality requirements; two vanishing moments and unit L2 norm by themselves do not supply those requirements.