Boole's inequality 2026-10-14
Boole's inequality, also called the union bound, bounds the probability that at least one of several events occurs by adding their individual probabilities. For a finite or countable family of events in the same probability space,The union includes outcomes where several events occur together. There is no independence assumption: adding the individual probabilities counts such outcomes repeatedly, which can only make the sum larger. The cap at one uses the total mass of a probability measure. For infinitely many events, the sum is a nonnegative series, possibly infinite; an infinite sum gives only the bound one. This is countable subadditivity of a measure specialized to a probability measure. See Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(ii), printed p. 2).
For example, choose uniformly from the twelve outcomes , using the discrete uniform distribution. Put . If , these are disjoint events and their union has probability , exactly the union bound. If , the union has only six outcomes, so its exact probability is , although both individual probabilities are still and the bound is still . If , the exact probability falls to with the same bound. These examples isolate overlap in the union bound by keeping the individual probabilities fixed.
The same individual event probabilities give different exact union probabilities as overlap increases
. Each panel is a fully specified discrete uniform distribution on twelve numbered outcomes. Blue means only, orange means only, purple means their intersection, and white means neither. The bottom bars compare the exact union probability with the sum of the individual probabilities. The three unions contain eight, six, and four outcomes; the sum counts eight in every panel. Details.The union bound is useful when each event is easy to control but their joint behavior is difficult to determine. A short proof uses indicator functions: at every outcome,Taking expected values and using linearity of expectation proves the finite bound.[ref] For a countable family, the finite unions increase to the full union; continuity of probability passes the finite inequalities to the limit. The closure of a sigma-algebra under countable unions ensures that this limiting union is an event. For the continuity of probability step, see Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(iii), printed p. 2).
Borel-Cantelli first lemma Created 2026-10-06 Updated 2026-10-14
For arbitrary events in one probability space,Equivalently, only finitely many events occur almost surely; no independence is needed.[ref]
Proof outline: put . The union bound givesThe tail of a convergent nonnegative series tends to zero.
Borel-Cantelli second lemma 2026-10-14
If the events are mutually independent events and , then . For an infinite family, mutual independence means every finite subfamily satisfies the factorization defining independent events.[ref]
Proof outline: for , independent events and giveContinuity of probability makes each infinite tail union have probability one; their countable intersection has probability one as well.
Brownian motion Created 2026-09-24 Updated 2026-10-14
Standard one-dimensional Brownian motion is a real-valued stochastic process on a probability space with three defining properties: almost surely; its sample paths are continuous almost surely; and its Brownian increments over any finite collection of disjoint time intervals are independent, withHere denotes the normal distribution with expectation and variance . The alternative symbol denotes the same kind of stochastic process. Continuity is a simultaneous statement about an entire sample path, rather than a separate assertion at each fixed time. Relative to its natural Brownian filtration, a future Brownian increment is independent of all observations up to its starting time.
The normalization gives , so the standard deviation of a Brownian increment over duration is . For , , and , the corresponding standard deviations are , , and . Time becoming one hundred times shorter makes typical displacement ten times smaller. Brownian scaling expresses this rule for the whole stochastic process, while the Brownian covariance kernel describes how positions at different times remain related despite independent increments.
For a concrete probability calculation, let be the cumulative distribution function of . At time four, has that normal distribution, henceThese numbers can be reproduced using the error function: . In particular, zero expectation does not mean small displacement: , but the same fixed interval contains only about of the probability at time four.
The probability density function of , for , isIts partial derivatives satisfyThus the normal distribution spreads according to the heat equation. Starting at a fixed position replaces by . This is a useful bridge between individual random trajectories and the deterministic evolution of their probability density function: the same Brownian motion provides both descriptions.
Physical diffusion uses its own scale. For constant diffusion coefficient , the position model has variance , and its probability density function satisfies . For example, gives displacement variance after four seconds, hence standard deviation . Standard Brownian motion corresponds numerically to in chosen units; omitting the factor two changes the physical convention. Einstein's 1905 paper relates diffusion to the mean squared displacement in precisely this convention. [ref]
https://djalil.chafai.net/docs/M2/history-brownian-motion/Einstein%20-%201905.pdf. Einstein, 1905 paper, English translation, section 4
Continuity does not supply a smooth physical velocity. The difference quotient has normal distribution , so its standard deviation grows as : for the three durations above it is , , and . This calculation motivates the nowhere differentiability of Brownian motion, but it alone is not a proof of the simultaneous assertion at every time. The theorem says that, with probability one, no finite derivative exists anywhere along the sample path. A drawn polygon only displays a finite grid; its straight segments cannot establish this theorem. See Theorem 7.1.6 in Probability: Theory and Examples (fifth edition) by Rick Durrett.
For physical particles, Brownian motion is a model of position on suitable observation scales. It does not claim that microscopic velocity is a derivative of this idealized path. Einstein explicitly restricts his assumption of independent increments to intervals that are not too short. [ref]
https://djalil.chafai.net/docs/M2/history-brownian-motion/Einstein%20-%201905.pdf. Einstein, 1905 paper, English translation, section 4
Another way to measure the roughness is quadratic variation. On , divide time into equal intervals and formWriting each Brownian increment as with independent standard normal random variables gives . Since and , the variance of is , andThis proves mean-square convergence to along uniform partitions. For and , the expectation is , the variance is , and the root mean square of the error is . For a general deterministic partition of an interval with lengths , the same calculation gives error variance , so shrinking the largest interval also suffices.
The resulting quadratic variation is , whereas the smooth function gives . This difference explains why the ordinary rules of calculus need the second-order correction in the Itô formula. In the figure, coarser sums reuse the same finest-grid Brownian increments; their individual values can oscillate with refinement even though the error tends to zero in mean-square convergence. Refining a fixed interpolating polygon forever would instead give zero quadratic variation, so the finite-grid illustration must not be confused with the limiting Brownian motion.
Historically, Brown's 1828 report describes microscopic observations made in 1827, starting with particles released from pollen and extending to inorganic materials; this challenged an explanation based solely on living matter. His observations and the later mathematical model are distinct contributions. [ref]
https://en.wikisource.org/wiki/The_miscellaneous_botanical_works_of_Robert_Brown/Volume_1/A_brief_account_of_microscopical_observations_made_on_the_particles_contained_in_the_pollen_of_plants. Brown, 1828 report, reprinted in his collected works
Einstein's 1905 paper derived a quantitative connection between kinetic theory, diffusion, and displacement. [ref]https://djalil.chafai.net/docs/M2/history-brownian-motion/Einstein%20-%201905.pdf. Einstein, 1905 paper, English translation
Wiener's 1923 Differential-Space developed integration on spaces of paths, a foundation for Wiener measure; it also acknowledges earlier work and his own previous papers. These dates mark documented contributions rather than an assertion that each subject had a single discoverer. [ref]https://djalil.chafai.net/docs/M2/history-brownian-motion/Wiener%20-%201923.pdf. Wiener, Differential-Space, 1923, sections 1 and 6
Brownian scaling Created 2026-09-24 Updated 2026-10-14
For every , the stochastic process is standard Brownian motion. Its Brownian increments remain independent and have variance ; continuity and the zero starting value are preserved. Equivalently,Equality here is of whole-process distributions, including every finite-dimensional distribution. It is not an equality of the two expressions along a particular realization. With , speeding time up fourfold requires dividing displacement by two to recover the standard law. The figure applies exactly these coordinate changes to four finite samples; the shared colors indicate reused samples, while the dashed lines show pointwise standard deviations, not bounds on entire sample paths.
Seeded Gaussian samples before and after Brownian scaling with c equal to four
. Details. The same square-root normalization explains how a simple random walk approaches Brownian motion. Let , where the independent steps take values and with equal probability. Then has variance . Its linear interpolation isand . Donsker's theorem says in with the uniform norm: this is convergence in distribution of random paths. The interpolation bound vanishes, but the theorem does not assert pointwise convergence of an arbitrarily fixed walk to an independently chosen Brownian motion. The usefulness is that many-step discrete motion has a universal continuous limit.[ref]
Characteristic function Created 2026-09-24 Updated 2026-10-18
The characteristic function of a real random variable isIt determines the probability distribution of .
Here and . By Euler's formula, its real part and imaginary part are and . The complex exponential has absolute value one, so this expected value exists for every real random variable, even without finite absolute moments. Moreover,The bound uses the triangle inequality for a vector-valued integral; the last identity is complex conjugate symmetry.[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. §3.3.1 definition, printed p. 125; Theorem 3.3.1(a)–(c), printed p. 126
Consequently the real part is an even function and the imaginary part is an odd function.For a discrete probability distribution with values and probabilities , the characteristic function is : a weighted average of points on the unit circle in the complex plane. For example, if has the Bernoulli distribution with and , thenEach point being averaged has absolute value one, but their directions partially cancel. At , the same average is ; at , it returns to one. The characteristic function is thus neither a probability density function nor a probability mass function: its argument is a frequency parameter, and its values can be complex numbers. This Bernoulli distribution has no probability density function with respect to Lebesgue measure, yet its characteristic function is defined at every frequency.
For a probability density function , the same definition becomes , the Fourier transform with the displayed sign convention. The Laplace transform of a nonnegative random variable instead uses for , a real decaying weight. For the Bernoulli distribution above, it is ; neither transform is the probability density function. The uniqueness theorem for characteristic functions explains why the oscillating averages still retain the entire probability law, while the characteristic function of a sum of independent variables turns combining laws into multiplication. These two properties make the representation useful even when directly computing a sum's probability distribution is cumbersome.
If is an integer and , the characteristic function has continuous derivatives through order , withThus finite absolute moments justify recovering moments from derivatives.[ref]
For a short justification of differentiation under the integral sign, the difference quotient for the st derivative has integrand . Its absolute value is at most , since for real . Lower absolute moments are finite because . Apply the dominated convergence theorem successively, then use the same integrable bound to obtain continuity of each derivative.
For the Bernoulli distribution with parameter , direct differentiation gives and , agreeing with . Its variance is therefore . Existence and continuity of a characteristic function alone do not authorize these moment calculations: the standard Cauchy distribution, whose expected value is undefined, still has a characteristic function. The stated finite absolute moment hypothesis is a sufficient condition for the derivative formula, not an assumption built into the definition of a characteristic function.
Characteristic function of a sum of independent variables Created 2026-09-24 Updated 2026-10-18
Independence makes the characteristic function of a sum equal the product of the individual characteristic functions.
For mutually independent random variables , write . ThenThis is the expectation of a product of independent random variables applied to bounded complex exponentials; no moment hypothesis is needed.[ref]
Identical probability distributions are unnecessary. When the random variables are independent and identically distributed random variables, the product is . A deterministic rescaling also follows directly from the definition: for real . Consequently the sample mean has characteristic function , and a sum divided by has characteristic function .
The independence assumption does real work. Let take and with equal probabilities and set . Both characteristic functions are , but , not . At they give and zero respectively. Knowing the two marginal probability distributions does not determine the sum without their joint probability distribution.
Continuity of a characteristic function Created 2026-09-29 Updated 2026-10-18
Every characteristic function is uniformly continuous and satisfies and . Continuity follows from the dominated convergence theorem applied to .
More explicitly, if , then pointwise and . Apply the dominated convergence theorem to the real part and imaginary part separately. There is no finite moment hypothesis. For uniform continuity, the stronger estimatefollows by factoring out , whose absolute value is one. The integrand tends pointwise to zero and is bounded by two, so the dominated convergence theorem applies again. The bound is independent of , which supplies uniform continuity on the entire real line.[ref]
If the first absolute moment is finite, additionally gives , a Lipschitz continuity estimate. The earlier argument matters because continuity survives even when this expected value is infinite. For convergence in distribution, the separate Lévy continuity theorem concerns limits of whole sequences of characteristic functions; the argument here establishes continuity of one characteristic function.
Kolmogorov's zero–one law 2026-10-14
For mutually independent random variables, every tail event has . Identical probability distributions are unnecessary. The tail sigma-algebra is trivial in probability: its events have probability zero or one, though it can contain nonempty proper sets.[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. Probability: Theory and Examples (fifth edition), §2.5, Theorem 2.5.3, printed pp. 81–82
For a proof, set and . Independence extended from generating pi-systems makes independent of , hence of each tail event . The increasing union is a generating pi-system for . For fixed , the events satisfying form a Dynkin system. The pi-lambda theorem extends this equality to all . Taking givesThus is independent of itself, forcing the two stated values.[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. Probability: Theory and Examples (fifth edition), proof of Theorem 2.5.3, steps (a) and (b), printed pp. 81–82
The useful feature is the absence of intermediate probabilities for infinite properties that ignore every finite prefix. It reduces some calculations to showing that a tail event has positive probability, or that its complement does. The theorem supplies no finite observation deadline, and an extra argument such as the calculation in infinitely many heads as a tail event is needed to decide which value occurs.
Limsup of events 2026-10-14
For events in a probability space, the infinitely-often event isIts complement of an event is : after some outcome-dependent index, no more events occur.[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. §2.3 opening definition and infinitely-often notation, printed pp. 67–68
Deleting finitely many events leaves the limsup of events unchanged. It is therefore a tail event of the sequence of indicator random variables , whether or not those random variables are independent. For independent events, the Kolmogorov zero-one law restricts its probability to zero or one; the Borel-Cantelli lemmas determine which alternative holds from the series of probabilities. Also precisely when only finitely many events occur, connecting the First Borel-Cantelli lemma to almost sure convergence and the existing complete convergence criterion.
Let be an integrable random variable, let be a finite -measurable random variable, and suppose is an integrable random variable. Then a known factor can be taken outside the conditional expectation:For a bounded , first verify the defining integral identity when is the indicator function of a measurable set in , extend by linearity to simple functions, and then use bounded approximation and the dominated convergence theorem. For nonnegative , truncating and using the conditional monotone convergence theorem proves the identity with extended nonnegative conditional expectations. In particular,For general under the stated integrability assumptions, the conditional Jensen inequality bounds by that integrable random variable. Apply the bounded-factor result to truncations of , then use the dominated convergence theorem on the products and the L1 contraction of conditional expectation. This proves the identity without assuming itself is an integrable random variable.
The product-integrability condition is essential to the ordinary finite conditional expectation statement. An unbounded factor cannot be moved through an expected value simply because its partner is an integrable random variable.[ref]
Put . Both and its indicator function are measurable, and . Thus is an integrable random variable. Monotonicity of conditional expectation givesThe measurable-factor property of conditional expectation says that a bounded -measurable function satisfies for every integrable random variable . Here the factor need not be bounded. Apply that property first to and , then use monotone convergence on the nonnegative products, including the conditional version, to obtainAll the left-hand conditional expectations are well defined; the final one is finite almost surely because is an integrable random variable. Since almost surely, division now yieldsThis proves the conditional Markov inequality with a measurable threshold without assuming that is an integrable random variable. The measurable-factor rule, with its product-integrability condition, is also stated in the standard conditional expectation theory.[ref]
The L2-bounded martingale convergence theorem states that a martingale with converges both almost surely and with convergence in L2 to a square-integrable random variable. Its hypotheses hold because . Thus there is almost surely with both kinds of convergence. Alternatively, the martingale convergence theorem gives the almost sure convergence, and the bound gives convergence in L2 by the dominated convergence theorem.[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. §4.4, Theorem 4.4.6, printed p. 237: the Lp martingale convergence theorem
The calculation of the conditional expectation of the square is especially informative. The martingale property makes the cross term vanish in the square expansion, soSubtracting this from givesTaking expected values and iterating provesBy bounded convergence theorem, . The random variable inside this expected value is nonnegative, so it is zero almost surely; on this forces . Again by bounded convergence theorem, . HenceThis is the Bernoulli polarization martingale with update size . It also has an exact mean squared error: the terminal representation and the equality case for conditional second moments give
The tail sigma-algebra of this sequence isFor each fixed , the preceding event equality givesEvery event on the right is determined by coordinates with index at least . Therefore for every , and . The random variables are independent, so Kolmogorov's zero–one law applies: every tail event has probability zero or one. Hence .
The role of independence can also be seen directly. A tail event is independent of each finite initial sigma-algebra . The Monotone class theorem extends this independence to ; since itself lies there, it is independent of itself. Thus .[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. §2.5, Theorem 2.5.3, printed pp. 81–82: the tail sigma-field zero–one law
For , the bound ensures the finite moment . By Jensen's inequality, . The process is an adapted, nonnegative integrable random variable at each time, and independence givesThus is a submartingale, and . The Doob maximal inequality for a nonnegative submartingale states that for . Apply it with . Since the power function is increasing on ,Combining this with the preceding bound givesThe result is a bound on the running maximum, which is why a maximal inequality for a submartingale, rather than an estimate of the single terminal random variable, is needed.[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. §4.4, Theorem 4.4.2, printed p. 235: the submartingale maximal inequality
Consider the Borel setIts indicator function is nonnegative and measurable, so the Tonelli theorem applies to the product measure, without needing any separate absolute-integrability argument. At a fixed , its section in the first coordinate is ; outside that section is empty. ThereforeThe last equality uses the definition of a cumulative distribution function on the half-closed interval. It retains all point atoms of a measure on the diagonal , excludes a possible point atom of a measure at , and includes , exactly as required by the set . If , both sides vanish. This verifies the identity even when the probability measures are not non-atomic measures.[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. §1.7, Theorem 1.7.2, printed pp. 38–40: integration of a nonnegative measurable function on a product space
The product measure is the unique probability measure on the sigma-algebra generated by measurable rectangles for whichHere the printed product notation denotes the generated product sigma-algebra, conventionally written ; it is not just the collection of rectangles. On it equals : coordinate projections show that rectangles are Borel sets, while open rectangles with rational endpoints give every open set as a countable union and hence generate the Borel sigma-algebra.
Existence is the product measure theorem for finite measures, which are in particular sigma-finite measures. Uniqueness follows because the rectangles form a pi-system generating the product sigma-algebra, and two finite measures agreeing there agree everywhere by the Dynkin lemma.[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. §1.7, Theorem 1.7.1, printed p. 38: existence and characterization on rectangles
The optional stopping theorem at a bounded stopping time states that an integrable martingale satisfies whenever for a fixed finite . Apply this first to at . The simple symmetric random walk cannot cross an integer endpoint without hitting it, soThe preceding part gives almost surely, hence almost surely. The constant bound permits the dominated convergence theorem, yielding . Since is either or , solving gives its full probability distribution:Now letThis counts the visit at time , and excludes the exit time. Apply the same bounded-time optional stopping theorem to . For each it givesThe visit counts increase to , so monotone convergence applies to the left. Dominated convergence applies to the right using . Consequently is an integrable random variable andThis is the origin case of expected occupation before interval exit for a simple symmetric random walk. The argument obtains integrability of the total occupation count before using any unbounded-time optional stopping for , so it does not presuppose the uniform integrability asked about next. The endpoint probabilities also agree with the standard gambler's ruin calculation.[ref]
Probability: Theory and Examples (fifth edition) by Rick Durrett. §4.8.1, Theorem 4.8.7(a), printed pp. 257–258: symmetric random-walk interval-exit probabilities




