Extreme value theorem Created 2026-09-24 Updated 2026-10-14
A real-valued function that is continuous on a nonempty compact space attains both a global minimum and a global maximum. In particular, for a continuous function on a nonempty closed interval, there are points such that
The important word is “attains”: the best values occur at actual allowed inputs, rather than merely being approached. The interval version is [ref].
For a worked example, take on . Its global minimum is , and its global maximum is . Nonnegativity gives the first answer; on the specified closed interval gives the second. The endpoint is essential: looking only for a zero derivative would find the minimum and miss the maximum.
Figure 1.
An attained minimum and maximum on a closed interval, compared with excluded endpoints and a discontinuous endpoint value
. Details.
For comparison, the continuous function on the open interval has infimum and supremum , but neither is a value of . Even a closed domain is insufficient without continuity: define for , and . On , its supremum is , approached by for , but every function value is strictly less than . This endpoint change makes discontinuous, preventing attainment. On the unbounded real line, the continuous function has neither a global minimum nor a global maximum. These examples show what can fail when a hypothesis is dropped; functions outside the hypotheses may still have extrema.
The extreme value theorem is an existence tool for optimization: first establish that a continuous objective function on a nonempty compact feasible set has a best value, then investigate where it occurs. It neither locates the optimizing point nor promises uniqueness or differentiability. For example, the absolute value function on has its global minimum at zero although no derivative exists there; a constant function has every input as a minimizer. This role in optimization is explained in University of Toronto MAT237 notes, section 1.4: compactness and the extreme value theorem.
A short proof outline identifies the mechanism. The continuous image of a compact space is compact; a nonempty compact subset of the real line contains its infimum and supremum. For a closed interval, one can instead choose inputs whose outputs approach the supremum, extract a convergent subsequence by the Bolzano-Weierstrass theorem, and use continuity to make its limiting output equal that supremum. The closed interval retains the limiting input. Repeating the argument for the infimum supplies the minimum.
Fenchel–Young gap Created 2026-10-05 Updated 2026-10-14
The Fenchel–Young gap is the nonnegative discrepancy
for a proper convex function and its convex conjugate. It can be . When is finite, it is the vertical distance between and its affine minorant . The Fenchel–Young gap vanishes exactly when , so it measures whether an input and a slope satisfy the subgradient relation.
For the convex quadratic function , completing the square gives
At and , the function value is and the tangent line has value , giving a Fenchel–Young gap of . In , the same calculation for gives and , half the squared Euclidean norm of the mismatch.
Figure 1. . The left panel fixes and marks the distance between the convex quadratic function and its tangent line . The right panel plots the Fenchel–Young gap , which vanishes at the matching input . Details.
An exact optimization error certificate. Fix a slope with and minimize the linearly tilted objective function . By the definition of the convex conjugate, , so
This identity holds even when the infimum is not attained. Thus a computed finite Fenchel–Young gap is exactly the error in objective value for this minimization problem. For the convex quadratic function, implies . A general proper convex function need not convert a small gap into a small distance: the absolute value has zero gap at for every .
The Fenchel–Young gap also connects convex analysis to optimal transport. For a proper convex function , define Kantorovich potentials and . The half-squared Euclidean distance cost satisfies . For a transport plan with finite integrated cost and integrable Kantorovich potentials, integrating this identity gives the difference between transport cost and the potentials' bound. This is the same local subgradient mismatch summed over transported pairs.
Fenchel–Young inequality Created 2026-09-28 Updated 2026-10-14
The Fenchel–Young inequality says that every input and every slope obey
where is any proper extended-real function and is its convex conjugate. No convexity assumption on is needed for this inequality: the defining supremum is at least the particular value . If either term on the left is , the inequality still holds, but cannot be an equality because the dot product on the right is finite.
For a proper convex function, equality is equivalent to the subgradient condition
To see this, with finite, equality says that attains the supremum defining , so for every . Rearranging gives the subgradient inequality . Geometrically, the affine minorant touches at .
If the proper convex function is also closed, meaning its epigraph is a closed set, or equivalently it is lower semicontinuous, then
The forward implication holds without closedness; the reverse one requires the additional hypothesis in general. Applying the equality criterion to relates to , and the Fenchel-Moreau theorem supplies when is closed. See Bertsekas, Convex Optimization Theory: A Summary, §5.4.1, Proposition 5.4.3 for precisely this distinction.
A concrete failure of the reverse implication occurs for the indicator functional of a constraint set of the open interval . Its convex conjugate is : the endpoint gives the supremum, even though it is excluded from the interval. At , , but , so equality fails and is empty. Closing the interval changes the endpoint value and restores equality there.
For the convex quadratic function , the Fenchel–Young inequality is , and equality holds exactly when . For the absolute value , finite equality requires at , where is the sign function; at every gives equality. These are exactly the subgradients of the absolute value, including its entire interval of slopes at the corner.
For the geometric series with terms , , the partial sums are . Averaging the finite expressions gives
If and , the second term has absolute value at most , and therefore tends to zero. On the unit circle, except at , this yields a first-order Cesàro sum even though the terms do not tend to zero. For , instead and , so there is no finite Cesàro sum.[ref]
Fixed-sign null space condition Created 2026-10-05 Updated 2026-10-14
For a real vector with support , uniqueness in basis pursuit is equivalent to for every nonzero . The supporting-line inequality for the absolute value proves sufficiency. Taking small positive and negative multiples of before any active sign changes proves necessity.
Frequency deviation 2026-10-14
The peak frequency deviation is the largest absolute value of the instantaneous frequency offset from the chosen carrier wave frequency . For , it is . It is a frequency, measured in hertz, rather than a wave phase or an exact signal bandwidth.
Gaussian elimination with a row swap Created 2026-10-18 Updated 2026-10-20
Consider this original linear system and its augmented matrix:
The first diagonal entry is zero, so it cannot serve as a pivot element. Interchange and to obtain a pivot element of one:
This chooses a convenient nonzero entry for exact arithmetic; partial pivoting would instead interchange and , selecting the largest absolute value, two. For our chosen ordering, the first-column elimination multipliers are and . The elementary row operation gives
The next pivot element is two and the next elimination multiplier is . Applying gives the upper triangular matrix system
Every operation also changed the right-hand side. Back substitution now gives
Checking the original equations, rather than only the reduced ones, verifies the answer by matrix multiplication:
Thus a zero first diagonal entry was a row-order obstruction, not a singular matrix. Indeed, the diagonal product of the final upper triangular matrix is ; the single row interchange makes , since the two row additions preserve the determinant.
Figure 1.
A row interchange supplies the first pivot element; successive elementary row operations create zeros, then back substitution gives
. Details.
Gershgorin circle theorem Created 2026-09-24 Updated 2026-10-18
The Gershgorin circle theorem locates every eigenvalue of a square matrix using only its entries. A diagonal entry supplies a centre in the complex plane, and the sum of the off-diagonal absolute values in its row supplies a radius. Small off-diagonal couplings therefore confine the eigenvalues near the diagonal entries. This turns a global eigenvalue problem into a picture built one row at a time.
Precisely, let , with , and define the closed Gershgorin discs
Every eigenvalue of satisfies , including eigenvalues on the boundaries. No symmetry or diagonalizability is required. Despite the name, the theorem uses filled disks, not just their circular boundaries.[ref]
The short reason is the triangle inequality. For a nonzero eigenvector with , choose with . The th row gives
Thus the largest eigenvector coordinate identifies a Gershgorin disc containing , even though that eigenvector need never be computed to use the bound.
This is useful when a full eigenvalue computation would be expensive: forming the Gershgorin discs needs one pass over the nonzero entries of a sparse matrix. If lies outside every Gershgorin disc, then is an invertible matrix, since zero is not an eigenvalue. The condition is exactly in every row, namely strict diagonal dominance. This connects diagonal dominance to solvability of a linear system; the inverse infinity-norm bound from diagonal dominance adds a quantitative bound. It is a sufficient test: failure to exclude zero does not prove singularity.
For a concrete symmetric matrix, take
The three Gershgorin discs have centres and are pairwise disjoint. The finite-dimensional spectral theorem makes the eigenvalues real, so their possible locations reduce to the intervals , , and . Gershgorin eigenvalue counting gives exactly one eigenvalue in each. Independently, the characteristic polynomial is
whose numerical polynomial roots are , , and , rounded to six decimal places. All are positive, as Gershgorin bounds for symmetric matrices already guarantee without finding those polynomial roots.
Figure 1.
Three separated Gershgorin discs, each containing one real eigenvalue of the symmetric example
. Details.
S. Gerschgorin published the result in 1931 in Über die Abgrenzung der Eigenwerte einer Matrix, in the Bulletin de l'Académie des Sciences de l'URSS, issue 6, pages 749–754. The original paper's §2 states both eigenvalue inclusion and counting in separated groups of Gershgorin discs; §3 already discusses column bounds and transformations preserving the eigenvalues. Thus both the localization picture and ways to sharpen it were part of the original development.[ref]
Global convergence of a square-root iteration Created 2026-10-18 Updated 2026-10-31
For every , this recurrence relation converges to the unique nonnegative fixed point . If , then , since . If , then . Thus the iterates form a monotone bounded sequence, and continuity forces their limit to be . A cobweb plot alternates vertical steps to and horizontal steps to , approaching the same intersection from either side. More quantitatively,
so the error keeps its sign and decreases in absolute value by at least the uniform factor . The asymptotic factor is .
Globally Lipschitz function Created 2026-09-28 Updated 2026-10-14
A function is globally Lipschitz when one finite Lipschitz bound holds for every pair in its entire specified domain of a function. The sine and the absolute value have this property on the real line. For any reference point , a globally Lipschitz continuous real function obeys , so its size grows at most linearly in the distance from .
The function is a locally Lipschitz function on the real line, but is not globally Lipschitz there: pairs , have ratio , which is unbounded. Restricting the domain to does give a global bound . Global and local describe where one bound must work, rather than two intrinsic labels independent of the domain.
Half-normal distribution Created 2026-09-24 Updated 2026-10-14
The half-normal distribution describes the absolute value of a centered Gaussian random variable. Its unit-scale version is for a standard normal random variable . It folds the negative half of the standard normal distribution onto the positive half, retaining the size of a deviation while discarding its sign. If a centered Gaussian random variable models measurement error, its magnitude therefore has a half-normal distribution.
Here is the standard normal density and is the standard normal distribution function. For , the event is , so the symmetric probability distribution of gives . The upper-tail probability is therefore . For example, . The doubled probability density function integrates to one on , and .
The mean and variance of the unit-scale half-normal distribution are
For a short derivation, because , while gives . More generally, for has a half-normal distribution with probability density function for . Its mean is and its variance is ; here is the scale of the underlying Gaussian random variable, not the standard deviation of its absolute value.
The ideal class group of is . Its four reduced binary quadratic forms have coefficients , , and . The displayed multiquadratic field has degree four over the base. The seven quadratic-subfield discriminants are , whose product in absolute value is . The conductor-discriminant formula and discriminant tower formula imply that the relative discriminant over is the unit ideal. This unramified abelian degree-four extension is therefore its Hilbert class field.
Initial value problem Created 2026-10-06 Updated 2026-10-14
An initial value problem combines an ordinary differential equation with an initial condition, for example and . The task is to find a solution of a differential equation on an interval containing which has that starting value. For a continuous linear differential equation , every prescribed finite vector determines a unique solution on the coefficient interval. A boundary value problem can instead prescribe values at distinct endpoints.
For the cooling initial value problem , , where and are constant, the unique solution is
Take degrees Celsius, degrees Celsius, , and per minute. Then degrees Celsius. Reaching degrees Celsius requires , so minutes. The time constant minutes reduces the excess temperature by a factor ; it does not reduce the absolute temperature by that factor.
Uniqueness needs assumptions. If is a continuous function on an open neighborhood of and satisfies a local Lipschitz condition in , uniformly in , the Picard-Lindelöf theorem gives a unique local solution. Concretely, a finite bounds in a sufficiently small rectangle. Cooling satisfies this with . Local existence does not promise existence for all future time: , , has on and finite-time blowup at .
Continuity alone is insufficient for uniqueness. Consider the dimensionless initial value problem involving a square root and an absolute value:
Besides , each delay gives a solution for :
At , delays give , despite identical starting data. These are valid solutions of a differential equation: for , both one-sided derivatives at the join are zero; for , the right derivative at the starting time is zero. After the join, because . Squaring alone would also suggest a parabola before the join, but its negative derivative there would violate the equation. Since , a forward solution starting from zero stays nonnegative.
Figure 1.
One initial value permits infinitely many delayed-start solutions without a Lipschitz condition
. Details.
The right-hand side is a continuous function but fails the Lipschitz condition at zero: for , is unbounded. Thus the Peano existence theorem still supplies local existence, while the Picard-Lindelöf theorem does not supply uniqueness. Related waiting-time examples and a sufficient uniqueness condition appear in MIT 18.034, Lecture 2: Fundamental Principles.
A brief proof outline explains the mechanism: rewrite the initial value problem as the integral equation . On a short enough interval, the Lipschitz condition makes the right-hand side a contraction mapping on a closed set of continuous functions near . The contraction mapping theorem gives a unique fixed point; the fundamental theorem of calculus turns it back into a solution.
Integer Created 2026-09-24 Updated 2026-10-18
An integer is a whole number, positive, negative, or zero. The set of all integers is
The positive integers are ; the negative integers are their opposites . Zero is neither positive nor negative. The nonnegative integers include zero and all the positive integers. On a number line, consecutive integers are one unit apart, negative integers lie to the left of zero, and positive integers lie to its right. [ref]
The integers let counting describe changes in both directions. If a counter starts at and increases by five, it reaches : . Decreasing that result by four gives . In addition on the number line, the sign of the added integer chooses the direction and its absolute value gives the distance. The additive inverse reverses the change made by .
Figure 1. . Starting at , a change of ends at . A subsequent change of ends at . The arrows indicate changes; the labelled points indicate integers. Zero separates the negative and positive sides. Details.
The integers have closure under a binary operation for addition, subtraction and multiplication: whenever , the results , and are still integers. Together these operations give the integers their ring structure.[ref] For instance, , , and . General division has no such closure: is a rational number but not an integer. Some divisions are exact, such as ; division by zero is undefined. Euclidean division instead produces an integer quotient and a remainder, so retains the information that ordinary division puts into the fraction .
An integer is the value, whereas its written numeral is an encoding. The decimal representation and the binary representation name the same integer. Changing positional notation changes how that value is displayed and how calculations are organized; it does not change the value itself.
Every occurs as the radius of convergence of a power series all of whose coefficients are positive integers. For , use the ceiling function to choose . Then
For , the comparison test for series with a geometric series proves absolute convergence. For , the bound violates the fact that terms of a convergent series tend to zero. Hence the radius of convergence is exactly .
For , use : at every the ratio of successive term absolute values is , eventually exceeding two. The factorial coefficients therefore give divergence away from zero.
A power series with every coefficient a positive integer cannot have radius of convergence greater than one: makes convergence impossible at . If zero coefficients are allowed and only finitely many are nonzero, a polynomial instead has infinite radius of convergence.
Integrable singularity 2026-10-18
A function has an integrable singularity at a point if it becomes unbounded there but its absolute value has a finite improper integral over a sufficiently small surrounding interval, one-sided at an endpoint. The endpoint behavior of the arcsine distribution's probability density function is an example.
Every nonnegative integer is a sum of squares with four terms, with zero terms allowed. Precisely, for every integer , there exist nonnegative integers such that
Allowing arbitrary integer roots gives the same statement: changing a root to its absolute value leaves the resulting square number unchanged. Four is a uniform upper bound on the number of nonzero terms, rather than a demand for four positive terms. Repeated square numbers are allowed. Conrad states the positive case and explicitly explains the role of zero terms; follows by taking all four roots zero. [ref]
The small cases make the convention visible:
In particular, could not be a sum of squares with four strictly positive roots, since each term would be at least one. Geometrically, a square of integer side length has area : the theorem says that any nonnegative whole-number total area can be split into at most four such square areas. It does not assert that those squares tile one larger square or a prescribed rectangle.
Figure 1.
Four-square representations as areas counted in unit cells
. The roots for were found by enumerating nonnegative four-tuples. Each panel has the same unit length, and each little cell has area one. The four separate squares illustrate addition of areas, with no claim about tiling a prescribed shape. These are exact arithmetic examples, not measured data. Details.
Why does the fourth term matter? Two square numbers do not suffice for : in modular arithmetic, a square number has remainder zero or one modulo four, so their sum cannot be three modulo four. The sum of two squares theorem gives a complete criterion for the positive integers representable with two terms. Even three terms leave gaps. A square number has remainder or modulo eight; adding three of these, with repetition allowed, never gives seven modulo eight. Thus cannot be sums of three square numbers, although Lagrange's four-square theorem represents all of them with four. This also proves that the bound four cannot be reduced to three.
The modulo-eight obstruction is not the only way that three terms can fail. For example, needs four: if three square numbers added to , their remainders modulo four would force all three roots even. Dividing by four would then express as a sum of three square numbers, contradicting the preceding calculation. These short checks explain the examples without requiring a full classification of sums of three square numbers.
Figure 2.
Minimum number of nonzero square terms for integers from one to one hundred
. Each cell is labelled by the positive integer being represented; its colour records the least number of nonzero square numbers whose sum equals it. Direct enumeration gives requiring one term, two, three, and four. Zero is omitted from the grid and requires no nonzero terms. Zero padding converts every representation into four slots. This finite computation illustrates Lagrange's four-square theorem; it does not prove its assertion for all nonnegative integers. Details.
The practical benefit of Lagrange's four-square theorem is an existence guarantee for a family of Diophantine equations: for every nonnegative integer , the displayed equation has an integer solution. It does not specify a unique solution. For instance, . To find a representation by a finite search, enumerate and test the sum. The floor function supplies a finite bound because no single term can exceed , and sorting roots removes duplicate orderings without discarding any possible sum. The theorem guarantees that the search succeeds.
A greedy algorithm that repeatedly subtracts the largest available square number need not find four terms: for it chooses , while uses four. The appeal of the theorem is this gap between local choices and a global promise: individual square numbers become farther apart, and two or three terms leave arithmetic obstructions, yet four terms cover every nonnegative integer.
For historical orientation, Conrad dates Lagrange's four-square theorem to and identifies the memoir as Démonstration d'un théorème d'arithmétique: it appeared in the Berlin Academy's volume for the year , published in , pages –. Keeping the volume year and publication year distinct avoids an apparent conflict between dates. [ref]
Least common multiple Created 2026-09-24 Updated 2026-10-20
For positive integers , their least common multiple is the smallest positive integer that is an integer multiple of both. Write it as . A positive common integer multiple always exists, since is one, but it can be larger than necessary.
For example, the positive integer multiples of begin , while those of begin . Their first positive coincidence is , so . Both also divide zero, but the word positive excludes zero from this minimum.
Suppose two events happen every twelve and eighteen days, and both happen at day . Their next meeting is at day , after three twelve-day intervals and two eighteen-day intervals. Through day , their common event times are .
Figure 1.
Events every 12 and 18 days, starting together, meet again every 36 days
. Each row marks the nonnegative integer multiples of its interval from zero through 72. The highlighted columns are shared; 36 is the first positive one and hence the least common multiple. Details.
The prime factorization explains why this coincidence occurs. For the integers and ,
A common integer multiple must contain at least the prime valuations required by each input. Taking the larger prime valuation at each prime number meets both requirements with no extra factors. The greatest common divisor uses the smaller prime valuations instead. This is the same coordinate comparison developed under prime factorization, applied in opposite directions.
Figure 2.
For 12 and 18, smaller prime valuations give greatest common divisor 6 and larger ones give least common multiple 36
. Bars show the exponents of the prime numbers two and three in , , and . The table makes the minimum and maximum in each column explicit. Details.
For any positive integers , the prime valuations give
The proof outline is one observation at each prime number: . Hence the two sides have identical prime factorizations. Numerically, . This also gives a calculation using the Euclidean algorithm for the greatest common divisor, without first finding the prime factors of both inputs.
Every positive common integer multiple is an integer multiple of the least common multiple, because its prime valuations must be at least the larger input prime valuations. Including negatives and zero, the common integer multiples of and are exactly for integers . For more than two positive integers, the same definition and rule of taking the largest prime valuations apply; one may also combine inputs successively. For example, .
For nonzero signed integers, take absolute values: . We extend the definition to zero by the convention , since zero is then the only common integer multiple; this is an extension, not a positive minimum. In particular must be handled separately from the displayed quotient, which would involve division by zero.
The least common multiple is useful for combining fractions with the smallest positive shared denominator. For example,
Its scheduling interpretation requires care about starting times. If the events start at integer days and , a common time must satisfy the integer congruences and . The Chinese remainder theorem for noncoprime moduli says a solution exists exactly when . If it exists, solutions form one congruence class modulo , but the first meeting need not be day . With starts , the first meeting is day and subsequent meetings are ; with , they never meet. The beauty of the least common multiple is the exact shared spacing, while the integer congruences determine whether and where the spacing begins.
Book VII, Proposition 34 of Euclid's Elements constructs the least common multiple of two numbers, and Proposition 36 extends the construction to three by first combining two of them. In modern notation this is . These are specific ancient treatments of the construction, rather than a claim about its first discovery. [ref] [ref]
Limit of a sequence Created 2026-10-06 Updated 2026-10-14
A sequence of real numbers has limit of a sequence , written , precisely when
The absolute value measures the error. The tolerance is chosen first; may depend on it, but must control every later term. Infinitely many accurate terms alone do not suffice. For example, repeatedly equals one but also repeatedly differs from one by two. The definition and uniqueness of the limit of a sequence are discussed in MIT 18.100A, Lecture 6, definitions 13 and theorem 15.
For a worked error estimate, let . Then , so any positive integer works. With , choose ; every later error is at most . The strict inequality matters: would give equality at its first term. A limit of a sequence turns the informal instruction “take enough terms” into an explicit accuracy guarantee.
The limit of a sequence is unique: two distinct proposed answers cannot both be within arbitrarily small error of every late term, by the triangle inequality. Changing finitely many initial terms does not change it. For an extended real number, means ; convergence to similarly requires . These describe escape rather than a finite answer. The extended-real Fekete lemma is an example where the limit of a sequence need not be finite.
L-infinity norm 2026-10-14
On a measure space of positive total measure, the L-infinity norm of an essentially bounded measurable function is . The absolute value matters: a constant function has L-infinity norm three, even though its essential supremum is . The norm belongs to the equivalence class in Lp space; different representatives equal almost everywhere give the same answer. [ref].
For the function in essential infimum and essential supremum, under Lebesgue measure, whereas its supremum norm is four. Replacing the exceptional values by the ordinary values leaves every finite Lp norm unchanged as well: . This tends to one as . That agreement explains why the endpoint with exponent uses an essential bound rather than a pointwise bound.
A finite probability space makes the distinction between zero and small measure explicit. Let , , and . Assign weights , , , where . For ,
When , the singleton is a null set and . For every , however small, . For example, gives and , yet the large- Lp norms approach ten. The ordinary pointwise supremum norm is ten in both cases.
Figure 1.
A zero-weight value disappears from every Lp norm, while a rare positive-weight value determines the limiting L-infinity norm
. The original three-point example has values . The curves use weights and , respectively. Dashed levels show the limiting norms three and ten; the plotted finite exponents do not themselves reach those limits. Details.
Why can a rare value dominate? For a bounded function on a probability space, an almost everywhere bound gives . If , the set has positive probability, so . As , the factor tends to one. These two bounds squeeze the limit to . The probability measure assumption keeps total measure finite; this explanation does not assert the same finite- integrability on arbitrary infinite-measure spaces.