Boole's inequality 2026-10-14
Boole's inequality, also called the union bound, bounds the probability that at least one of several events occurs by adding their individual probabilities. For a finite or countable family of events in the same probability space,
The union includes outcomes where several events occur together. There is no independence assumption: adding the individual probabilities counts such outcomes repeatedly, which can only make the sum larger. The cap at one uses the total mass of a probability measure. For infinitely many events, the sum is a nonnegative series, possibly infinite; an infinite sum gives only the bound one. This is countable subadditivity of a measure specialized to a probability measure. See Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(ii), printed p. 2).
For example, choose uniformly from the twelve outcomes , using the discrete uniform distribution. Put . If , these are disjoint events and their union has probability , exactly the union bound. If , the union has only six outcomes, so its exact probability is , although both individual probabilities are still and the bound is still . If , the exact probability falls to with the same bound. These examples isolate overlap in the union bound by keeping the individual probabilities fixed.
Figure 1.
The same individual event probabilities give different exact union probabilities as overlap increases
. Each panel is a fully specified discrete uniform distribution on twelve numbered outcomes. Blue means only, orange means only, purple means their intersection, and white means neither. The bottom bars compare the exact union probability with the sum of the individual probabilities. The three unions contain eight, six, and four outcomes; the sum counts eight in every panel. Details.
The union bound is useful when each event is easy to control but their joint behavior is difficult to determine. A short proof uses indicator functions: at every outcome,
Taking expected values and using linearity of expectation proves the finite bound.[ref] For a countable family, the finite unions increase to the full union; continuity of probability passes the finite inequalities to the limit. The closure of a sigma-algebra under countable unions ensures that this limiting union is an event. For the continuity of probability step, see Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(iii), printed p. 2).
Borel-Cantelli first lemma Created 2026-10-06 Updated 2026-10-14
For arbitrary events in one probability space,
Equivalently, only finitely many events occur almost surely; no independence is needed.[ref]
Proof outline: put . The union bound gives
The tail of a convergent nonnegative series tends to zero.
If the events are mutually independent events and , then . For an infinite family, mutual independence means every finite subfamily satisfies the factorization defining independent events.[ref]
Proof outline: for , independent events and give
Continuity of probability makes each infinite tail union have probability one; their countable intersection has probability one as well.
Brownian motion Created 2026-09-24 Updated 2026-10-14
Standard one-dimensional Brownian motion is a real-valued stochastic process on a probability space with three defining properties: almost surely; its sample paths are continuous almost surely; and its Brownian increments over any finite collection of disjoint time intervals are independent, with
Here denotes the normal distribution with expectation and variance . The alternative symbol denotes the same kind of stochastic process. Continuity is a simultaneous statement about an entire sample path, rather than a separate assertion at each fixed time. Relative to its natural Brownian filtration, a future Brownian increment is independent of all observations up to its starting time.
The normalization gives , so the standard deviation of a Brownian increment over duration is . For , , and , the corresponding standard deviations are , , and . Time becoming one hundred times shorter makes typical displacement ten times smaller. Brownian scaling expresses this rule for the whole stochastic process, while the Brownian covariance kernel describes how positions at different times remain related despite independent increments.
For a concrete probability calculation, let be the cumulative distribution function of . At time four, has that normal distribution, hence
These numbers can be reproduced using the error function: . In particular, zero expectation does not mean small displacement: , but the same fixed interval contains only about of the probability at time four.
The probability density function of , for , is
Its partial derivatives satisfy
Thus the normal distribution spreads according to the heat equation. Starting at a fixed position replaces by . This is a useful bridge between individual random trajectories and the deterministic evolution of their probability density function: the same Brownian motion provides both descriptions.
Physical diffusion uses its own scale. For constant diffusion coefficient , the position model has variance , and its probability density function satisfies . For example, gives displacement variance after four seconds, hence standard deviation . Standard Brownian motion corresponds numerically to in chosen units; omitting the factor two changes the physical convention. Einstein's 1905 paper relates diffusion to the mean squared displacement in precisely this convention. [ref]
Continuity does not supply a smooth physical velocity. The difference quotient has normal distribution , so its standard deviation grows as : for the three durations above it is , , and . This calculation motivates the nowhere differentiability of Brownian motion, but it alone is not a proof of the simultaneous assertion at every time. The theorem says that, with probability one, no finite derivative exists anywhere along the sample path. A drawn polygon only displays a finite grid; its straight segments cannot establish this theorem. See Theorem 7.1.6 in Probability: Theory and Examples (fifth edition) by Rick Durrett.
For physical particles, Brownian motion is a model of position on suitable observation scales. It does not claim that microscopic velocity is a derivative of this idealized path. Einstein explicitly restricts his assumption of independent increments to intervals that are not too short. [ref]
Another way to measure the roughness is quadratic variation. On , divide time into equal intervals and form
Writing each Brownian increment as with independent standard normal random variables gives . Since and , the variance of is , and
This proves mean-square convergence to along uniform partitions. For and , the expectation is , the variance is , and the root mean square of the error is . For a general deterministic partition of an interval with lengths , the same calculation gives error variance , so shrinking the largest interval also suffices.
The resulting quadratic variation is , whereas the smooth function gives . This difference explains why the ordinary rules of calculus need the second-order correction in the Itô formula. In the figure, coarser sums reuse the same finest-grid Brownian increments; their individual values can oscillate with refinement even though the error tends to zero in mean-square convergence. Refining a fixed interpolating polygon forever would instead give zero quadratic variation, so the finite-grid illustration must not be confused with the limiting Brownian motion.
Historically, Brown's 1828 report describes microscopic observations made in 1827, starting with particles released from pollen and extending to inorganic materials; this challenged an explanation based solely on living matter. His observations and the later mathematical model are distinct contributions. [ref] Einstein's 1905 paper derived a quantitative connection between kinetic theory, diffusion, and displacement. [ref] Wiener's 1923 Differential-Space developed integration on spaces of paths, a foundation for Wiener measure; it also acknowledges earlier work and his own previous papers. These dates mark documented contributions rather than an assertion that each subject had a single discoverer. [ref]
Brownian scaling Created 2026-09-24 Updated 2026-10-14
For every , the stochastic process is standard Brownian motion. Its Brownian increments remain independent and have variance ; continuity and the zero starting value are preserved. Equivalently,
Equality here is of whole-process distributions, including every finite-dimensional distribution. It is not an equality of the two expressions along a particular realization. With , speeding time up fourfold requires dividing displacement by two to recover the standard law. The figure applies exactly these coordinate changes to four finite samples; the shared colors indicate reused samples, while the dashed lines show pointwise standard deviations, not bounds on entire sample paths.
Figure 1.
Seeded Gaussian samples before and after Brownian scaling with c equal to four
. Details.
The same square-root normalization explains how a simple random walk approaches Brownian motion. Let , where the independent steps take values and with equal probability. Then has variance . Its linear interpolation is
and . Donsker's theorem says in with the uniform norm: this is convergence in distribution of random paths. The interpolation bound vanishes, but the theorem does not assert pointwise convergence of an arbitrarily fixed walk to an independently chosen Brownian motion. The usefulness is that many-step discrete motion has a universal continuous limit.[ref]
Characteristic function Created 2026-09-24 Updated 2026-10-18
The characteristic function of a real random variable is
It determines the probability distribution of .
Here and . By Euler's formula, its real part and imaginary part are and . The complex exponential has absolute value one, so this expected value exists for every real random variable, even without finite absolute moments. Moreover,
The bound uses the triangle inequality for a vector-valued integral; the last identity is complex conjugate symmetry.[ref] Consequently the real part is an even function and the imaginary part is an odd function.
For a discrete probability distribution with values and probabilities , the characteristic function is : a weighted average of points on the unit circle in the complex plane. For example, if has the Bernoulli distribution with and , then
Each point being averaged has absolute value one, but their directions partially cancel. At , the same average is ; at , it returns to one. The characteristic function is thus neither a probability density function nor a probability mass function: its argument is a frequency parameter, and its values can be complex numbers. This Bernoulli distribution has no probability density function with respect to Lebesgue measure, yet its characteristic function is defined at every frequency.
Figure 1.
A Bernoulli characteristic function as a weighted average on the unit circle
. At , the two complex exponentials are and . Left: their weighted average lies one third of the way along their connecting segment. Right: adding the weighted arrows and gives . The dashed unit circle makes the absolute bound one visible. Details.
For a probability density function , the same definition becomes , the Fourier transform with the displayed sign convention. The Laplace transform of a nonnegative random variable instead uses for , a real decaying weight. For the Bernoulli distribution above, it is ; neither transform is the probability density function. The uniqueness theorem for characteristic functions explains why the oscillating averages still retain the entire probability law, while the characteristic function of a sum of independent variables turns combining laws into multiplication. These two properties make the representation useful even when directly computing a sum's probability distribution is cumbersome.
If is an integer and , the characteristic function has continuous derivatives through order , with
Thus finite absolute moments justify recovering moments from derivatives.[ref]
For a short justification of differentiation under the integral sign, the difference quotient for the st derivative has integrand . Its absolute value is at most , since for real . Lower absolute moments are finite because . Apply the dominated convergence theorem successively, then use the same integrable bound to obtain continuity of each derivative.
For the Bernoulli distribution with parameter , direct differentiation gives and , agreeing with . Its variance is therefore . Existence and continuity of a characteristic function alone do not authorize these moment calculations: the standard Cauchy distribution, whose expected value is undefined, still has a characteristic function. The stated finite absolute moment hypothesis is a sufficient condition for the derivative formula, not an assumption built into the definition of a characteristic function.
Characteristic function of a sum of independent variables Created 2026-09-24 Updated 2026-10-18
Independence makes the characteristic function of a sum equal the product of the individual characteristic functions.
For mutually independent random variables , write . Then
This is the expectation of a product of independent random variables applied to bounded complex exponentials; no moment hypothesis is needed.[ref]
Identical probability distributions are unnecessary. When the random variables are independent and identically distributed random variables, the product is . A deterministic rescaling also follows directly from the definition: for real . Consequently the sample mean has characteristic function , and a sum divided by has characteristic function .
The independence assumption does real work. Let take and with equal probabilities and set . Both characteristic functions are , but , not . At they give and zero respectively. Knowing the two marginal probability distributions does not determine the sum without their joint probability distribution.
Continuity of a characteristic function Created 2026-09-29 Updated 2026-10-18
Every characteristic function is uniformly continuous and satisfies and . Continuity follows from the dominated convergence theorem applied to .
More explicitly, if , then pointwise and . Apply the dominated convergence theorem to the real part and imaginary part separately. There is no finite moment hypothesis. For uniform continuity, the stronger estimate
follows by factoring out , whose absolute value is one. The integrand tends pointwise to zero and is bounded by two, so the dominated convergence theorem applies again. The bound is independent of , which supplies uniform continuity on the entire real line.[ref]
If the first absolute moment is finite, additionally gives , a Lipschitz continuity estimate. The earlier argument matters because continuity survives even when this expected value is infinite. For convergence in distribution, the separate Lévy continuity theorem concerns limits of whole sequences of characteristic functions; the argument here establishes continuity of one characteristic function.
For mutually independent random variables, every tail event has . Identical probability distributions are unnecessary. The tail sigma-algebra is trivial in probability: its events have probability zero or one, though it can contain nonempty proper sets.[ref]
For a proof, set and . Independence extended from generating pi-systems makes independent of , hence of each tail event . The increasing union is a generating pi-system for . For fixed , the events satisfying form a Dynkin system. The pi-lambda theorem extends this equality to all . Taking gives
Thus is independent of itself, forcing the two stated values.[ref]
The useful feature is the absence of intermediate probabilities for infinite properties that ignore every finite prefix. It reduces some calculations to showing that a tail event has positive probability, or that its complement does. The theorem supplies no finite observation deadline, and an extra argument such as the calculation in infinitely many heads as a tail event is needed to decide which value occurs.
Limsup of events 2026-10-14
For events in a probability space, the infinitely-often event is
Its complement of an event is : after some outcome-dependent index, no more events occur.[ref]
Deleting finitely many events leaves the limsup of events unchanged. It is therefore a tail event of the sequence of indicator random variables , whether or not those random variables are independent. For independent events, the Kolmogorov zero-one law restricts its probability to zero or one; the Borel-Cantelli lemmas determine which alternative holds from the series of probabilities. Also precisely when only finitely many events occur, connecting the First Borel-Cantelli lemma to almost sure convergence and the existing complete convergence criterion.
Let be an integrable random variable, let be a finite -measurable random variable, and suppose is an integrable random variable. Then a known factor can be taken outside the conditional expectation:
For a bounded , first verify the defining integral identity when is the indicator function of a measurable set in , extend by linearity to simple functions, and then use bounded approximation and the dominated convergence theorem. For nonnegative , truncating and using the conditional monotone convergence theorem proves the identity with extended nonnegative conditional expectations. In particular,
For general under the stated integrability assumptions, the conditional Jensen inequality bounds by that integrable random variable. Apply the bounded-factor result to truncations of , then use the dominated convergence theorem on the products and the L1 contraction of conditional expectation. This proves the identity without assuming itself is an integrable random variable.
The product-integrability condition is essential to the ordinary finite conditional expectation statement. An unbounded factor cannot be moved through an expected value simply because its partner is an integrable random variable.[ref]
Put . Both and its indicator function are measurable, and . Thus is an integrable random variable. Monotonicity of conditional expectation gives
The measurable-factor property of conditional expectation says that a bounded -measurable function satisfies for every integrable random variable . Here the factor need not be bounded. Apply that property first to and , then use monotone convergence on the nonnegative products, including the conditional version, to obtain
All the left-hand conditional expectations are well defined; the final one is finite almost surely because is an integrable random variable. Since almost surely, division now yields
This proves the conditional Markov inequality with a measurable threshold without assuming that is an integrable random variable. The measurable-factor rule, with its product-integrability condition, is also stated in the standard conditional expectation theory.[ref]
The L2-bounded martingale convergence theorem states that a martingale with converges both almost surely and with convergence in L2 to a square-integrable random variable. Its hypotheses hold because . Thus there is almost surely with both kinds of convergence. Alternatively, the martingale convergence theorem gives the almost sure convergence, and the bound gives convergence in L2 by the dominated convergence theorem.[ref]
The calculation of the conditional expectation of the square is especially informative. The martingale property makes the cross term vanish in the square expansion, so
Subtracting this from gives
Taking expected values and iterating proves
By bounded convergence theorem, . The random variable inside this expected value is nonnegative, so it is zero almost surely; on this forces . Again by bounded convergence theorem, . Hence
This is the Bernoulli polarization martingale with update size . It also has an exact mean squared error: the terminal representation and the equality case for conditional second moments give
The tail sigma-algebra of this sequence is
For each fixed , the preceding event equality gives
Every event on the right is determined by coordinates with index at least . Therefore for every , and . The random variables are independent, so Kolmogorov's zero–one law applies: every tail event has probability zero or one. Hence .
The role of independence can also be seen directly. A tail event is independent of each finite initial sigma-algebra . The Monotone class theorem extends this independence to ; since itself lies there, it is independent of itself. Thus .[ref]
For , the bound ensures the finite moment . By Jensen's inequality, . The process is an adapted, nonnegative integrable random variable at each time, and independence gives
Thus is a submartingale, and . The Doob maximal inequality for a nonnegative submartingale states that for . Apply it with . Since the power function is increasing on ,
Combining this with the preceding bound gives
The result is a bound on the running maximum, which is why a maximal inequality for a submartingale, rather than an estimate of the single terminal random variable, is needed.[ref]
Consider the Borel set
Its indicator function is nonnegative and measurable, so the Tonelli theorem applies to the product measure, without needing any separate absolute-integrability argument. At a fixed , its section in the first coordinate is ; outside that section is empty. Therefore
The last equality uses the definition of a cumulative distribution function on the half-closed interval. It retains all point atoms of a measure on the diagonal , excludes a possible point atom of a measure at , and includes , exactly as required by the set . If , both sides vanish. This verifies the identity even when the probability measures are not non-atomic measures.[ref]
The product measure is the unique probability measure on the sigma-algebra generated by measurable rectangles for which
Here the printed product notation denotes the generated product sigma-algebra, conventionally written ; it is not just the collection of rectangles. On it equals : coordinate projections show that rectangles are Borel sets, while open rectangles with rational endpoints give every open set as a countable union and hence generate the Borel sigma-algebra.
Existence is the product measure theorem for finite measures, which are in particular sigma-finite measures. Uniqueness follows because the rectangles form a pi-system generating the product sigma-algebra, and two finite measures agreeing there agree everywhere by the Dynkin lemma.[ref]
The optional stopping theorem at a bounded stopping time states that an integrable martingale satisfies whenever for a fixed finite . Apply this first to at . The simple symmetric random walk cannot cross an integer endpoint without hitting it, so
The preceding part gives almost surely, hence almost surely. The constant bound permits the dominated convergence theorem, yielding . Since is either or , solving gives its full probability distribution:
Now let
This counts the visit at time , and excludes the exit time. Apply the same bounded-time optional stopping theorem to . For each it gives
The visit counts increase to , so monotone convergence applies to the left. Dominated convergence applies to the right using . Consequently is an integrable random variable and
This is the origin case of expected occupation before interval exit for a simple symmetric random walk. The argument obtains integrability of the total occupation count before using any unbounded-time optional stopping for , so it does not presuppose the uniform integrability asked about next. The endpoint probabilities also agree with the standard gambler's ruin calculation.[ref]
Choose one random variable with the continuous uniform distribution on , and reuse it in every event:
Each probability is , so their harmonic series diverges, just as in summable and harmonic event probabilities. But the events are decreasing: . For every outcome , all indices with fail. Only finitely many events occur on that outcome. The limsup of events is exactly , a zero-probability event. This is a concrete closed-interval version of the shrinking-interval dependence example.[ref]
The missing assumption is visible already in two events. For , their intersection is , hence
Moreover, the whole tail union is just , with probability . This calculation explains directly why divergence alone fails: overlapping events keep spending their individual probabilities on the same outcomes.
Let be the total number of occurrences, a sum of indicator random variables. Although almost surely, Tonelli theorem gives . More explicitly, for : very small positive values of give very long, but finite, initial runs. Infinite expected value does not imply a positive probability of an infinite count.
Figure 1.
Equal harmonic event probabilities with fresh independent inputs or one shared input
. Each row is one finite simulation of the indicator random variables of for ; a dot means the event occurred. Fresh independent random variables allow scattered returns; a shared uniform random variable yields an initial run followed by zeros. The horizontal axis uses a logarithmic scale to show early runs clearly. These finite simulations illustrate the two models and cannot prove infinitely many occurrences or eventual stopping. Details.