Accumulation below an unattained supremum 2026-10-14
If a nonempty real set has finite supremum and , then every interval contains infinitely many elements of . Otherwise its finitely many elements have a largest value ; all remaining elements are at most , making an upper bound. The interval cannot be empty either, by the defining approximation property of a supremum. Thus failure to attain a finite supremum forces accumulation from below.
Anchored variation norm 2026-10-25
On the vector space of continuous functions of bounded variation with , the total variation of a function is a norm. Its triangle inequality and absolute homogeneity of a norm follow by applying the corresponding absolute value identities to every partition of an interval, then taking the supremum. The partition of an interval gives , so zero total variation of a function forces . The same inequality gives , proving that convergence for this norm implies uniform convergence. Without , every constant function would have zero total variation of a function.
Argmax 2026-11-05
The argmax of a function is the set of inputs where its largest value is attained:It records global maximizers, whereas records the value. For , the argmax on is and the maximum value is zero. If is constant, every input belongs to the argmax.
The argmax can be empty: on has supremum one but never attains it. A continuous function on a nonempty compact set does attain its maximum by the extreme value theorem. Writing a single input as an argmax therefore implicitly assumes uniqueness or specifies a rule for choosing between ties. This distinction matters when an optimization problem is used to define an estimator.
Argmin consistency under uniform convergence in probability Created 2026-10-06 Updated 2026-11-05
On a compact parameter set, a continuous deterministic objective with a unique minimum has a positive separation gap outside each neighbourhood of its minimizer. Uniform convergence in probability of the random objectives bounds the deterministic objective difference at any attained random minimizer by twice the supremum discrepancy. This proves statistical consistency of each measurable minimizing selection.
Dropping either condition destroys the implication. The increasing sequence has a lower bound one but no upper bound, and no finite limit of a sequence. For any real threshold , the Archimedean property gives an index , and monotonicity gives for all . Its divergence to in the extended real number line is different from convergence to a real number.
Conversely, is a bounded sequence, with , but is not a monotone sequence and has no limit of a sequence. Its subsequence at even integers is constantly one, and its subsequence at odd integers is constantly minus one. These two subsequential limits cannot both equal a single limit of a sequence. [ref]
Basic Analysis: Introduction to Real Analysis, Volume I by Jiří Lebl. §2.1, Example 2.1.5: divergence of the alternating sequence; §2.1.1, examples immediately after Definition 2.1.9: the increasing sequence n
Neither monotonicity alone nor boundedness alone ensures a finite limit of a sequence
. Left: is increasing but unbounded above. Right: stays in but alternates between two distinct subsequential limits. Both plots show finitely many terms; the formulas and the arguments establish the counterexamples for all indices. Details.The ambient real numbers also matter. An original example using the floor function isEvery is a rational number, because , andThe strict lower inequality uses the irrationality of the square root of two. Hence this bounded sequence converges to in the real numbers, but has no limit of a sequence in the rational numbers. Convergence to a rational candidate would also be convergence in the real numbers, contradicting uniqueness of the limit of a sequence. The formulas give , , , : this counterexample even has a plateau. This illustrates why completeness of the real numbers matters: the rational numbers lack the least-upper-bound property needed to contain the required supremum. [ref]
The supremum of an increasing bounded sequence need not be a maximum of its set of terms. For , every term is strictly less than one; the supremum and limit of a sequence are one, but there is no maximum. Every candidate term is beaten by the next. The decreasing sequence similarly has infimum one without a minimum.
If an increasing sequence with limit of a sequence attains , it must remain there: for . The decreasing case is identical with inequalities reversed. Thus attainment of the limit of a sequence is equivalent to an eventually constant tail of a sequence for a convergent monotone sequence. The increasing sequence has both supremum and maximum one. [ref]
Basic Analysis: Introduction to Real Analysis, Volume I by Jiří Lebl. §2.1, Exercise 2.1.13: a convergent monotone sequence attaining its limit
Bellman equation Created 2026-09-24 Updated 2026-10-14
A Bellman equation expresses the dynamic programming principle as a recurrence relation for a value function: optimize the current reward plus the best attainable continuation. Let be a dynamic programming state, the allowed actions at time , the immediate reward, and the next dynamic programming state. With terminal time and terminal reward , the deterministic finite-horizon Bellman equation isHere measures total reward from time through the terminal reward. Start with and evaluate backwards; the terminal condition anchors the recurrence relation. For finite nonempty action sets, finite rewards, and finite terminal rewards, a maximum exists at each step. If an optimum need not be attained, use a supremum. A minimization problem instead uses a minimum or infimum. The 0-1 knapsack problem above is the same comparison between an immediate item value plus a continuation and the continuation after skipping. See Bertsekas, MIT lecture 2: the backwards recursion.
In a Markov decision process, the next dynamic programming state is random. If denotes the expected immediate reward and is a discount factor, the finite-horizon Bellman equation becomesThe conditional expectation averages over the next dynamic programming state before choosing the current action; that action cannot use a future outcome that has not yet been observed. For a finite state space with transition probabilities , the continuation term is . Taking gives the undiscounted finite-horizon problem. With a discount factor , rewards steps ahead carry a factor , including the terminal reward. A maximizing action for each specifies a Markov policy, which contains decisions rather than just optimal scores.
For an infinite horizon with a finite state space, finite nonempty action sets, time-independent transition probabilities, bounded rewards , and a discount factor , the optimal value function satisfiesIn this setting the maximizing dynamic programming operator is a contraction mapping in the supremum norm, so the Banach fixed-point theorem gives a unique bounded fixed point and convergence of repeated application to it. See Bertsekas, MIT lecture 15: bounded discounted problems. These hypotheses matter: with one state, one action, zero reward, and , the Bellman equation is merely , so every finite constant is a fixed point, although the actual total-reward value function is zero. The average-reward Bellman equation and Hamilton-Jacobi-Bellman equation address other settings with their own conditions.
Borell-TIS inequality Created 2026-10-03 Updated 2026-11-05
Bounded monotone sequence theorem Created 2026-10-07 Updated 2026-10-20
A monotone sequence of real numbers has a finite limit of a sequence exactly when it is a bounded sequence. In the increasing case, only an upper bound needs to be supplied, because the first term already gives a lower bound; in the decreasing case, only a lower bound needs to be supplied. The limit of a sequence is the exact edge of the set of terms:The upper bound or lower bound need not itself be the limit of a sequence. A supremum or infimum identifies the sharp bound. [ref]
Basic Analysis: Introduction to Real Analysis, Volume I by Jiří Lebl. §2.1.1, Theorem 2.1.10 and the one-sided-boundedness observation following its proof
For example, is increasing and bounded above by one, and is decreasing and bounded below by one. Both have limit of a sequence one, although neither ever equals one. Their absolute errors are exactly . They also bracket any sequence satisfying : the squeeze theorem gives , even if itself is not a monotone sequence.
Increasing and decreasing sequences bracket a common limit of a sequence, with explicitly shrinking absolute errors
. Left: and enclose one; for every both lie strictly inside the tolerance band . Right: the absolute errors and shrink at different rates. The markers are individual terms, and connecting segments only guide the eye. Details.The theorem is useful when a recurrence relation gives successive improvements and a bound, before the answer has been identified. It proves existence of a limit of a sequence; further algebra, often using continuity, identifies the answer. A finite plot illustrates the formulas but does not establish the infinite claim. Error bounds for bounded monotone sequences require additional quantitative information.
A real increasing sequence bounded above converges to its supremum; a decreasing sequence bounded below converges to its infimum. For the increasing case, the definition of supremum provides a term above , and every later term remains between that term and . Negation reduces the decreasing case to the increasing case. This elementary result is distinct from the measure-theoretic monotone convergence theorem for integrals.
The existence of the real supremum in this short proof uses the least-upper-bound property, equivalently completeness of the real numbers. The approximation property of a supremum supplies , and monotonicity keeps every later term within the same tolerance:The corresponding decreasing statement follows from the approximation property of an infimum. The converse uses the fact that every convergent sequence is a bounded sequence. [ref]
Chaining Created 2026-10-25 Updated 2026-10-31
A multiscale method for bounding the supremum of a stochastic process. Approximate each index by points in successively finer metric nets, expand its difference from an anchor as a telescoping sum, and bound the largest increment at each scale. For a finite index set the finest net can be the entire set, making this decomposition exact. If is the set of approximation edges, then pointwise . Finite maxima controlled by moment-generating functions turn the sizes of the nets into metric entropy sums and integrals. The Dudley entropy integral and dyadic increment chaining are related instances.
Cofinal subset Created 2026-10-31 Updated 2026-11-05
A cofinal subset of a partially ordered set satisfies: every is at most some . For example, the even natural numbers are cofinal in the natural numbers, because every natural number is bounded above by an even one.
For a nonzero limit ordinal , this condition is equivalent to . For a successor ordinal , it instead requires : the singleton is already cofinal, although its supremum is . Thus the equality must not be used without the limit ordinal hypothesis. A convention valid for every ordinal is .
Completeness of the real numbers Created 2026-09-24 Updated 2026-10-14
Every Cauchy sequence of real numbers converges to a real number. Explicitly, the conditionimplies the existence of with . The hypothesis compares approximations with one another, without requiring a candidate answer. This makes completeness of the real numbers an existence principle: sufficiently consistent approximations cannot approach a missing destination. [ref]
https://ocw.mit.edu/courses/18-100a-real-analysis-fall-2020/mit18_100af20_lec102.pdf. MIT 18.100A, Lecture 10, theorem 7
.The equivalent least-upper-bound property says that every nonempty set bounded above has a supremum . Precisely, every satisfies , and for every some satisfies . The second condition excludes any smaller upper bound. No largest element of is required.
The rational numbers lack this property. The set contains one and has upper bound two, but has no supremum in . A proposed rational supremum would have to satisfy : if , a sufficiently small rational increase still lies in ; if , a sufficiently small rational decrease remains an upper bound. The irrationality of the square root of two excludes equality. In , its supremum is . [ref]
https://ocw.mit.edu/courses/18-100a-real-analysis-fall-2020/mit18_100af20_lec32.pdf. MIT 18.100A, Lecture 3, theorems 14–15
.Construct this square root by the bisection method, starting with rational bounds . At each step square the midpoint and retain the half whose endpoints bracket two after squaring. The first four resulting intervals are , , , and . After steps, and . Both endpoint sequences are Cauchy sequences because every later endpoint remains in the current interval. Completeness of the real numbers gives a common limit of a sequence ; continuity of squaring gives . The midpoint approximates with error at most . Thus exact rational arithmetic produces certified accuracy while completeness of the real numbers supplies the exact answer.
Richard Dedekind approached completeness of the real numbers by filling gaps in the rational numbers with Dedekind cuts. In the preface to his 1872 Continuity and Irrational Numbers, he traced the problem to teaching differential calculus in 1858 and wanting arithmetic foundations for limits. His construction makes a missing boundary, such as , into a number defined by the separation it produces among rational numbers. [ref]
https://www.gutenberg.org/files/21016/21016-pdf.pdf. Dedekind, Continuity and Irrational Numbers, preface and §§III–IV; Beman's 1901 translation
. Continuous-offer bargaining acceptance lemma 2026-10-31
Let a proposer divide a pie of size by continuously offering its responder . Rejection gives fixed continuation payoffs to the proposer and to the responder, with and . Then a subgame perfect equilibrium must have the responder accept with probability one. The proposer offers exactly , retaining .
Sequential rationality forces rejection for and acceptance for . If acceptance at had probability , that offer would give the proposer . Every larger offer gives less than and can be improved by a smaller offer still above ; every smaller offer gives . The supremum would therefore be unattained. Randomizing offers cannot attain it either: the nonnegative gap from would be strictly positive almost surely and hence have positive expectation. No best response would exist, contradicting equilibrium. Acceptance at removes this gap and makes the unique optimal proposal. The assumptions and exclude degenerate indifference of the proposer.
Convex conjugate Created 2026-09-28 Updated 2026-10-14
The convex conjugate records a function using slopes rather than input positions. Let be a proper extended-real function: it is finite somewhere and never takes . Its convex conjugate iswhere is the dot product and is the effective domain. An infinite value of excludes that input from the supremum. The value can be , but cannot be because an input with finite supplies a finite lower bound.
The convex conjugate is automatically a convex function, even when is not convex: each fixed input gives an affine function , and their supremum is convex. It is also lower semicontinuous. It need not be proper: for on , is unbounded above for every . The definition and the automatic convexity are discussed in Boyd and Vandenberghe, Convex Optimization (2004), §3.3.1.
The best lower bound with a prescribed slope. Whenever is finite, the affine functionis an affine minorant of . Indeed, an affine function lies below everywhere exactly when . Thus is the largest possible intercept at slope . The convex conjugate trades positions for slopes: specifies where to inspect , whereas specifies which global affine minorant to use. Taking the supremum over all these affine minorants produces the biconjugate; the Fenchel-Moreau theorem says this recovers a proper convex function when it is lower semicontinuous.
A quadratic, computed by completing the square. For the convex quadratic function ,Consequently , and the unique maximizing input is . The affine minorant is the tangent line at : its slope is and its intercept is . For example, gives and the tangent line , touching at . This is the same curve described either by its point values or by the intercepts of its tangent lines.
A corner turns into a constraint on slopes. For the absolute value , if then , with value zero at . If , taking makes ; if , taking does the same. Therefore its convex conjugate is zero on and outside: it is the indicator functional of a constraint set for this interval. For the unique maximizing input is zero; for every maximizes, and for every maximizes. The corner's many supporting slopes are exactly the subdifferential .
A supremum need not be attained. For a differentiable convex function finite on all of , an input maximizes exactly when . If is strictly convex, there is at most one such input; existence is a separate issue. For the exponential function , the derivative is positive. At , solving gives and . At , the supremum of is zero as , with no maximizing input. At , as . Thus a strictly convex differentiable function can have both unattained finite conjugate values and infinite conjugate values.
Counting measure Created 2026-09-29 Updated 2026-10-14
Counting measure makes integration into summation by giving every measurable singleton unit weight. On any measurable space , it is the measure defined byThus , a finite set has its usual cardinality, and every infinite set has the same value in the extended real number line. In particular, countably infinite sets and uncountable sets are not distinguished by their counting measure. Taking to be the power set makes every subset measurable; restricting the same rule to a smaller sigma-algebra is also allowed. [ref]
https://www.math.ucdavis.edu/~hunter/measure_theory/measure_notes_ch1.pdf. John K. Hunter, Measure Theory, Example 1.11: counting measure
Why is this a countably additive measure? For pairwise disjoint measurable sets ,If the union is finite, only finitely many are nonempty, and finite cardinality adds. If the union is infinite, either one is infinite or infinitely many are nonempty; in both cases the nonnegative series on the right is infinite. For example, and have counting measures and , and their union has counting measure . The singletons for positive natural numbers give the infinite case: .
With the power set as sigma-algebra, counting measure is a finite measure exactly when is finite, and a sigma-finite measure exactly when is at most countable. For the latter assertion, enumerate by measurable singletons; conversely, every finite-measure set is finite, and a countable union of finite sets is countable. Specifying the sigma-algebra matters: if and , counting measure is not a sigma-finite measure, since the only measurable finite-measure set is the empty set.
For a finite or countably infinite set equipped with its power set, every nonnegative function is a measurable function, andThe supremum allows an infinite result and avoids choosing an order for the terms. On a finite set, expand as a simple function using singleton indicator functions. On a countably infinite set, enumerate the points and apply the monotone convergence theorem to finite truncations. Real or complex functions are integrable functions for counting measure exactly when ; then their integral equals the absolutely convergent sum. This is the discrete counterpart of Lebesgue integration, developed further in series as a Lebesgue integral against counting measure. [ref]
https://www.math.ucdavis.edu/~hunter/measure_theory/measure_notes_ch4.pdf. John K. Hunter, Measure Theory, Example 4.10: absolutely convergent series as integrals
For example, on the positive natural numbers has counting measure integral one, by the geometric series. The alternating harmonic series has a finite ordered sum but is not the integral of an integrable function for counting measure: its positive and negative parts both have infinite sums. Absolute convergence is what makes the finite integral independent of enumeration. In particular, the Lp spaces for counting measure on are the l-p sequence spaces, with their sums replacing continuous integrals.
Compare three measures on the same Borel sigma-algebra of the real numbers: counting measure , Dirac measure , and Lebesgue measure .The rational numbers in the last row form a countably infinite set; their Lebesgue measure is zero by countable additivity applied to singletons, even though they are dense in the interval.
The same Borel sets have different sizes under counting measure, Dirac measure, and Lebesgue measure
. Original analytical comparison on the real numbers, with the selected point marked in orange. Drawings show the finite sets and the interval; the rational numbers are named symbolically because finitely many plotted points cannot depict that infinite dense set. Green zero entries mark the null sets for each column's measure. Details.These are different answers to different questions: counting measure asks how many points occur, Dirac measure asks whether the selected point occurs, and Lebesgue measure asks how much length occurs. A null set therefore depends on its measure. For counting measure, the only null set is the empty set, so equality almost everywhere is equality everywhere. For , a measurable set is a null set exactly when it misses ; for Lebesgue measure, even the infinite set is a null set. Integrating an indicator function recovers each table entry through . This lets the same Lebesgue integral framework handle point counts, point evaluation, and geometric length.
David Pollard 2026-11-05
David Pollard is a statistician whose research and exposition concern weak convergence, empirical processes and stochastic processes. Yale's institutional profile identifies these specialties, together with statistical asymptotics and econometrics.[ref]
A recurring scientific issue in his writing is how bounds for one random quantity become simultaneous bounds for a whole indexed family. His Measurability Woes by David Pollard explains why pointwise almost-sure agreement does not automatically provide a single exceptional set valid at every index. That distinction matters when a supremum or an entire sample path enters an argument. His author page lists Convergence of Stochastic Processes (1984) and Empirical Processes: Theory and Applications (1990), followed by broader notes on probability methods.[ref]
https://www.stat.yale.edu/~pollard/. Books and manuscripts
[ref] Dual norm Created 2026-10-06 Updated 2026-10-18
A dual norm measures how strongly a bounded linear functional can respond to a unit-sized input. If is a bounded linear functional on a normed vector space , write and defineIt is the operator norm of when its scalar output is measured by absolute value. On , represent using a coefficient vector . The unit ball is symmetric, so maximizing gives the same value as maximizing : negate an input whenever its output is negative.[ref]
Convex Optimization by Stephen Boyd and Lieven Vandenberghe. §A.1.6, printed p. 636: dual norm and scalar operator-norm interpretation
For an original two-coordinate example, fix , so . The input norm changes the allowable unit ball and hence the dual norm:Under the L1 norm, , and attains the bound. Under the Euclidean norm, the Cauchy-Schwarz inequality gives , with equality at the displayed normalized vector. Under the maximum norm, each coordinate can reach unit absolute value, so ; attains it. Thus the dual norms of these three vector norms are respectively the maximum norm, the Euclidean norm, and the L1 norm. For , Lp norms on a finite coordinate space pair with their Holder conjugate exponents.[ref]
Convex Optimization by Stephen Boyd and Lieven Vandenberghe. §A.1.6, printed p. 637: Euclidean self-duality and one-, infinity- and p-norm duality; the numerical vector and maximizing points here are independent examples
This geometry explains the usefulness of a dual norm: it translates a bound on inputs into the sharp bound on a linear output. The support function of the unit ball gives its value in the coefficient direction . Changing the input budget changes the set of possible inputs; the three different answers do not conflict. In finite dimensions a closed unit ball is compact, so a continuous linear functional achieves its maximum absolute value. In an infinite-dimensional normed vector space, the supremum remains the definition and need not be achieved.
Dual norm supremum need not be attained 2026-10-18
On the real space of sequences converging to zero , use the supremum norm and define the linear functionalThe geometric series gives , so is a bounded linear functional and . The sequence with its first entries equal to one and all later entries zero belongs to , has unit supremum norm, and gives . Hence . But if and , not all coordinates can equal one or all equal minus one. Since every weight is positive, . The supremum is approached but never attained on the unit ball.
Dudley entropy integral Created 2026-10-03 Updated 2026-11-05
Let be a centered Gaussian process with a separable version: its relevant suprema are captured by a countable dense index subset. Give the canonical pseudometric of a Gaussian process , and assume it is totally bounded. With metric covering number and diameter , the Dudley entropy integral boundsfor a universal constant . Finiteness of the integral gives a finite expected supremum. For an absolute supremum, include an anchor: . The anchor term cannot generally be omitted.
The proof builds successively finer nets. Write each increment as a telescoping sum between neighboring net scales, bound each finite Gaussian maximum by its scale times the square root of the logarithmic net size, and sum over scales. This becomes the displayed entropy integral.[ref]
The version assumption is substantive. If is uniform on and , every finite vector is zero almost surely and is therefore a degenerate Gaussian vector; the canonical pseudometric is identically zero. Yet for every outcome. Its all-zero separable modification has supremum zero. Finite-dimensional laws alone do not control the supremum of an arbitrary uncountable version.
Dvoretzky-Kiefer-Wolfowitz inequality 2026-10-18
For independent and identically distributed real random variables with cumulative distribution function , the Dvoretzky-Kiefer-Wolfowitz inequality gives, for every and ,The supremum is the Kolmogorov distance: a single event controls all thresholds simultaneously. The bound holds for discrete, continuous, and mixed probability distributions.[ref]
For one fixed threshold, Hoeffding's inequality gives the same expression as an upper bound on . That fixed-threshold result alone does not justify moving the supremum inside the probability. The DKW inequality is the stronger statement needed for a simultaneous empirical distribution confidence band.




