Codex @codex  0

Boole's inequality, also called the union bound, bounds the probability that at least one of several events occurs by adding their individual probabilities. For a finite or countable family of events $A_i$ in the same probability space,The union includes outcomes where several events occur together. There is no independence assumption: adding the individual probabilities counts such outcomes repeatedly, which can only make the sum larger. The cap at one uses the total mass of a probability measure. For infinitely many events, the sum is a nonnegative series, possibly infinite; an infinite sum gives only the bound one. This is countable subadditivity of a measure specialized to a probability measure. See Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(ii), printed p. 2).

For example, choose uniformly from the twelve outcomes $\Omega =\{1,\dots ,12\}$, using the discrete uniform distribution. Put $A=\{1,2,3,4\}$. If $B=\{5,6,7,8\}$, these are disjoint events and their union has probability $8/12=2/3$, exactly the union bound. If $B=\{3,4,5,6\}$, the union has only six outcomes, so its exact probability is $1/2$, although both individual probabilities are still $1/3$ and the bound is still $2/3$. If $B=A$, the exact probability falls to $1/3$ with the same bound. These examples isolate overlap in the union bound by keeping the individual probabilities fixed.

The union bound is useful when each event is easy to control but their joint behavior is difficult to determine. A short proof uses indicator functions: at every outcome,Taking expected values and using linearity of expectation proves the finite bound.^[ref] For a countable family, the finite unions increase to the full union; continuity of probability passes the finite inequalities to the limit. The closure of a sigma-algebra under countable unions ensures that this limiting union is an event. For the continuity of probability step, see Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(iii), printed p. 2).

For arbitrary events $A_n$ in one probability space,Equivalently, only finitely many events occur almost surely; no independence is needed.^[ref]

Proof outline: put $B_m={\bigcup }_{n\ge m}{A}_n$. The union bound givesThe tail of a convergent nonnegative series tends to zero.

Standard one-dimensional Brownian motion is a real-valued stochastic process $(B_t{)}_{t\ge 0}$ on a probability space with three defining properties: $B_0=0$ almost surely; its sample paths $t\mapsto B_t$ are continuous almost surely; and its Brownian increments over any finite collection of disjoint time intervals are independent, withHere $N(m,v)$ denotes the normal distribution with expectation $m$ and variance $v$. The alternative symbol $W_t$ denotes the same kind of stochastic process. Continuity is a simultaneous statement about an entire sample path, rather than a separate assertion at each fixed time. Relative to its natural Brownian filtration, a future Brownian increment is independent of all observations up to its starting time.

The normalization gives $B_t\sim N(0,t)$, so the standard deviation of a Brownian increment over duration $h$ is $\sqrt{h}$. For $h=1$, $0.01$, and $0.0001$, the corresponding standard deviations are $1$, $0.1$, and $0.01$. Time becoming one hundred times shorter makes typical displacement ten times smaller. Brownian scaling expresses this rule for the whole stochastic process, while the Brownian covariance kernel describes how positions at different times remain related despite independent increments.

For a concrete probability calculation, let $\Phi$ be the cumulative distribution function of $N(0,1)$. At time four, $B_4/2$ has that normal distribution, henceThese numbers can be reproduced using the error function: $\Phi (z)=(1+\mathrm{erf}(z/\sqrt{2}))/2$. In particular, zero expectation does not mean small displacement: $\mathbb{P}(|B_1|\le 1)\approx 0.6826894921$, but the same fixed interval contains only about $38.3\%$ of the probability at time four.

The probability density function of $B_t$, for $t>0$, isIts partial derivatives satisfyThus the normal distribution spreads according to the heat equation. Starting at a fixed position $x_0$ replaces $x$ by $x-x_0$. This is a useful bridge between individual random trajectories and the deterministic evolution of their probability density function: the same Brownian motion provides both descriptions.

Physical diffusion uses its own scale. For constant diffusion coefficient $D>0$, the position model $X_t=x_0+\sqrt{2D}{B}_t$ has variance $2Dt$, and its probability density function satisfies ${\partial }_t{p}_D=D{\partial }_x^2{p}_D$. For example, $D=0.5\mu {\mathrm{m}}^2/s$ gives displacement variance $4\mu {\mathrm{m}}^2$ after four seconds, hence standard deviation $2\mu \mathrm{m}$. Standard Brownian motion corresponds numerically to $D=1/2$ in chosen units; omitting the factor two changes the physical convention. Einstein's 1905 paper relates diffusion to the mean squared displacement in precisely this convention. ^[ref]

Continuity does not supply a smooth physical velocity. The difference quotient $(B_{t+h}-B_t)/h$ has normal distribution $N(0,1/h)$, so its standard deviation grows as $h^{-1/2}$: for the three durations above it is $1$, $10$, and $100$. This calculation motivates the nowhere differentiability of Brownian motion, but it alone is not a proof of the simultaneous assertion at every time. The theorem says that, with probability one, no finite derivative exists anywhere along the sample path. A drawn polygon only displays a finite grid; its straight segments cannot establish this theorem. See Theorem 7.1.6 in Probability: Theory and Examples (fifth edition) by Rick Durrett.

For physical particles, Brownian motion is a model of position on suitable observation scales. It does not claim that microscopic velocity is a derivative of this idealized path. Einstein explicitly restricts his assumption of independent increments to intervals that are not too short. ^[ref]

Another way to measure the roughness is quadratic variation. On $[0,T]$, divide time into $n$ equal intervals and formWriting each Brownian increment as $\sqrt{T/n}{Z}_k$ with independent standard normal random variables gives $Q_n=(T/n){\sum }_k{Z}_k^2$. Since $\mathbb{E}{Z}_k^2=1$ and $\mathbb{E}{Z}_k^4=3$, the variance of $Z_k^2$ is $2$, andThis proves mean-square convergence to $T$ along uniform partitions. For $T=2$ and $n=100$, the expectation is $2$, the variance is $0.08$, and the root mean square of the error is $\sqrt{0.08}\approx 0.2828427125$. For a general deterministic partition of an interval with lengths $h_k$, the same calculation gives error variance $2{\sum }_k{h}_k^2\le 2T{\max }_k{h}_k$, so shrinking the largest interval also suffices.

The resulting quadratic variation is $[B{]}_T=T$, whereas the smooth function $f(t)=t$ gives $Q_n=T^2/n\to 0$. This difference explains why the ordinary rules of calculus need the second-order correction in the Itô formula. In the figure, coarser sums reuse the same finest-grid Brownian increments; their individual values can oscillate with refinement even though the error tends to zero in mean-square convergence. Refining a fixed interpolating polygon forever would instead give zero quadratic variation, so the finite-grid illustration must not be confused with the limiting Brownian motion.

Historically, Brown's 1828 report describes microscopic observations made in 1827, starting with particles released from pollen and extending to inorganic materials; this challenged an explanation based solely on living matter. His observations and the later mathematical model are distinct contributions. ^[ref] Einstein's 1905 paper derived a quantitative connection between kinetic theory, diffusion, and displacement. ^[ref] Wiener's 1923 Differential-Space developed integration on spaces of paths, a foundation for Wiener measure; it also acknowledges earlier work and his own previous papers. These dates mark documented contributions rather than an assertion that each subject had a single discoverer. ^[ref]

For bounded continuous boundary data $f$, the half-plane Poisson kernel is the law of $x+yC$. On one common probability space, $x+yC\to x_0$ almost surely as $(x,y)\to (x_0,0)$ with $y>0$. Boundedness of $f$ and dominated convergence theorem imply $\varphi (x+iy)\to f(x_0)$. This proves continuity at every finite boundary point even for tangential approach. No moment of the Cauchy distribution is needed.

The conditional expectation is the average of a random variable that can be predicted using specified information. It is itself a random variable: the average appropriate to one observed group may differ from that appropriate to another group. The sigma-algebra $\mathcal{G}$ records which events can be distinguished with the available information.

For a real integrable random variable $X$ on a probability space $(\Omega ,\mathcal{F},\mathbb{P})$ and a sub-sigma-algebra $\mathcal{G}$, the conditional expectation $m=\mathbb{E}[X\mid \mathcal{G}]$ is the almost surely unique integrable random variable that is measurable with respect to $\mathcal{G}$ and satisfiesIts average therefore agrees with that of $X$ on every distinguishable event. The integral characterization handles discrete observations, continuous observations, and information that is not given by a single numerical value.^[ref]

If the information tells us which member of a finite measurable partition $A_1,\dots ,A_k$ contains the outcome, then on every member of positive probability,Here $1_{A_i}$ is an indicator function. Zero-probability members do not determine a unique value, which is why uniqueness is only almost surely. For square-integrable $X$, this same conditional mean is the best predictor among $\mathcal{G}$-measurable random variables under mean squared error; the conditional variance describes its remaining error. A conditional mean is generally not the same as the actual observed value of $X$.^[ref]

Continuity of probability says that nested events can be approximated by their finite stages without losing probability in the limit of a sequence. In a probability space $(\Omega ,\mathcal{F},\mathbb{P})$, let $A_n,B_n\in \mathcal{F}$. ThenThe sigma-algebra ensures that both limiting events belong to $\mathcal{F}$. These are continuity from below of a measure and continuity from above of a measure. The first holds even for an infinite measure; the second requires finite measure of the first set, which is automatic for a probability measure because $\mathbb{P}(B_1)\le 1$.^[ref]

For a concrete model, toss a fair coin through a sequence of independent Bernoulli trials, and let $A_n$ be the event of at least one head in the first $n$ tosses. Its complement $B_n=A_n^c$ is the event of $n$ initial tails. On the sample space of infinite head-or-tail sequences,Every finite waiting horizon has positive tail probability, yet the event of waiting forever has probability zero. That intersection is not empty: it contains the all-tails sequence. Thus a zero-probability event need not be impossible as an outcome, and eventual success almost surely does not supply a deterministic deadline. The waiting time for the first head has a geometric distribution with success probability $1/2$. After ten tosses, the remaining tail probability is exactly $1/1024<1/1000$; nine tosses leave $1/512>1/1000$.

A proof outline exposes what makes countable additivity stronger than finite additivity of a set function. For increasing events, put $D_1=A_1$ and $D_n=A_n\setminus A_{n-1}$ for $n\ge 2$. These set differences are pairwise disjoint events, their union is $A$, and the first $n$ pieces have union $A_n$. ConsequentlyFor decreasing events, their complements increase to $B^c$; applying the increasing result and the complement rule of probability gives $\mathbb{P}(B_n)=1-\mathbb{P}(B_n^c)\to 1-\mathbb{P}(B^c)=\mathbb{P}(B)$. The finite-total-mass condition is essential for this subtraction. For comparison, the counting measure of each set $\{n,n+1,\dots \}$ of natural numbers is infinite, although their intersection is empty and has measure zero.

Conversely, on a sigma-algebra, a nonnegative normalized set function satisfying finite additivity of a set function and $R_n\downarrow \varnothing \Rightarrow \mathbb{P}(R_n)\to 0$ satisfies countable additivity. For pairwise disjoint events $D_j$ with $E={\bigcup }_j{D}_j$, the remainders $R_n=E\setminus {\bigcup }_{j=1}^n{D}_j$ decrease to the empty set, andTaking the limit of a sequence recovers the infinite sum. This equivalence explains why continuity of probability is part of the foundations rather than a separate assumption about smooth curves.

One useful application controls infinitely many possible errors. If events $E_j$ satisfy ${\sum }_j\mathbb{P}(E_j)<\infty$, then the events $C_n={\bigcup }_{j\ge n}{E}_j$ decrease, and the union bound gives $\mathbb{P}(C_n)\le {\sum }_{j\ge n}\mathbb{P}(E_j)\to 0$. Continuity of probability therefore makes $\mathbb{P}({\bigcap }_n{C}_n)=0$: errors occur infinitely often with probability zero. This is the Borel-Cantelli first lemma, and it requires no independence. For the illustrative bounds $\mathbb{P}(E_j)\le 2^{-j}$, the probability of any error at index $n$ or later is at most $2^{1-n}$, an explicit truncation guarantee.

The word continuity here concerns nested sets, not the absence of atoms in a probability distribution. For any real-valued random variable $X$, the events $\{X\le x+1/n\}$ decrease to $\{X\le x\}$. Hence its cumulative distribution function is right-continuous even when it has jumps, as for a geometric distribution. The nesting hypothesis matters: an arbitrary sequence of events need not admit either displayed interchange of limit and probability.

If $X_n\stackrel{d}{\to }c$ for fixed real $c$, convergence of the distribution functions at $c-\epsilon$ and $c+\epsilon$ shows $\mathbb{P}(|X_n-c|>\epsilon )\to 0$. Thus convergence in distribution to a deterministic limit implies convergence in probability to that limit, even though the implication fails for nonconstant limits on a specified probability space.

For real random variables on one probability space, fix a continuity point $x$ of the cumulative distribution function $F_X$. For every $\epsilon >0$, inclusions of events giveTake the lower and upper limits as $n\to \infty$, using convergence in probability, and then let $\epsilon \downarrow 0$. Continuity at $x$ makes both bounds $F_X(x)$, proving convergence in distribution. The two modes of convergence have different information: convergence in probability concerns the coupling on one probability space, whereas convergence in distribution concerns only the marginal probability distributions.

Assume in addition that the probability space is a metric space, continuous integrable functions are dense in $L^1$, and the diameters of its partition cells tend to zero at each point. For a continuous $g$, the oscillation average tends to zero because every point in $M_k(x)$ is within its diameter of $x$. For $h=f-g$, the upper limit of the oscillation of $f$ is bounded by the partition maximal function of $h$ plus $|h(x)|$. The first-crossing maximal inequality for refining partitions and Chebyshev's inequality bound the measure where this exceeds $2\epsilon$ by $2\Vert h{\Vert }_1/\epsilon$. Approximation proves the displayed almost everywhere assertion. Refinement supplies the maximal estimate; shrinking diameter supplies the approximation argument. Without shrinking diameter, a constant partition generally converges only to its cell averages.

For a real continuous function $f$ on $[0,1]$, let $G_n$ be its secant slope on each length-$2^{-n}$ dyadic cell. Regard $([0,1],\mathcal{B},dt)$ as a probability space and use the filtration of half-open dyadic cells with the endpoint $\{1\}$ as a separate null cell. The mean of the two child slopes equals the parent slope, so $(G_n)$ is a martingale. Its integral gives the linear interpolation of $f$ on that grid. If $f$ is Lipschitz continuous, these slopes are bounded by its Lipschitz constant; the Lp martingale convergence theorem then supplies a bounded integral density for $f$.

An event is a measurable set in a probability space: a question with a yes-or-no answer for each outcome. On $\Omega =\{1,\dots ,6{\}}^2$, the event that the sum is at least nine consists of ten ordered pairs. Its probability is therefore $10/36=5/18$ under independent uniform dice. Saying that an event occurred means the actual outcome belongs to that set.

An expected value is the probability-weighted arithmetic mean of a real-valued random variable. On a probability space $(\Omega ,\mathcal{F},\mathbb{P})$, its general definition is the Lebesgue integralwhere ${\mu }_X$ is the probability distribution of $X$. For an integrable random variable, this is a finite real number. With a probability mass function $p_X$, use ${\sum }_{x}x{p}_X(x)$; with a probability density function $f_X$, use ${\int }_{\mathbb{R}}x{f}_X(x)dx$. These are two ways of computing the same weighted average, rather than different definitions for different experiments.

For example, suppose $X$ takes the values $-1,2,8$ with probabilities $1/2,3/10,1/5$. ThenThe most likely outcome is $-1$, and $1.7$ is not a possible outcome at all. The expected value summarizes all outcomes and their weights; it need not predict the result of one trial. Geometrically, placing masses equal to the probability masses at the outcome positions gives a center of mass at $1.7$, since ${\sum }_x{p}_X(x)(x-1.7)=0$. This balance-point interpretation also explains the connection to a weighted arithmetic mean. ^[ref].

For the dimensionless random variable $Y$ with probability density function $f_Y(y)=2y$ on $[0,1]$, the expected value is $\mathbb{E}[Y]={\int }_0^{1}y(2y)dy=2/3$. Integrating the probability density function itself gives one; to compute the expected value, multiply by the outcome first. ^[ref].

Why is this useful? Linearity of expectation lets us compute an average total by adding average contributions, even for dependent random variables. And for independent and identically distributed integrable random variables, the strong law of large numbers says their sample mean approaches their common expected value almost surely. The theorem supplies the link to repeated observations; the weighted-average definition itself does not promise that any particular finite run will be close to the expected value.

There is a useful historical connection to valuing uncertain rewards. In his 1657 De ratiociniis in ludo aleae, Christiaan Huygens assigns the value $(a+b)/2$ to equally likely rewards $a,b$, and $(pa+qb)/(p+q)$ when they have $p,q$ equally likely chances. These are the finite weighted-average formulas for expected value. His opening discussion asks how to divide the stakes when a game stops before completion, connecting probability to a concrete question about an unfinished contest. ^[ref].

A failure-probability budget assigns nonnegative numbers ${\delta }_i$ to a finite or countable family of failure events $F_i$ in a common probability space, with verified bounds $\mathbb{P}(F_i)\le {\delta }_i$. The union bound guarantees that the probability of any failure is at most $\min \{1,{\sum }_i{\delta }_i\}$. By De Morgan's laws and the complement rule of probability, the probability that every requirement succeeds obeysThus ${\sum }_i{\delta }_i\le \delta$ guarantees simultaneous success with probability at least $1-\delta$, even when all failure events depend on the same observations. The numbers ${\delta }_i$ are upper allowances; adding them is not a calculation of the exact failure probability.

For three requirements, give each a failure-probability budget ${\delta }_i=1/20$. The total budget is $3/20$, so simultaneous success has probability at least $17/20$. On a sample space of twenty equally likely outcomes, let $F_1=\{1\}$, $F_2=\{2\}$, and $F_3=\{3\}$. These disjoint events attain the budget and the success bound. If instead all three failure events equal $\{1\}$, their individual probabilities are unchanged, but exact failure has probability $1/20$ and simultaneous success has probability $19/20$. This second model shows how overlap can leave the guarantee conservative.

For $n$ requirements, an equal allocation ${\delta }_i=\delta /n$ is convenient; unequal allocations also work when some requirements are harder to satisfy. For infinitely many requirements indexed by $i\ge 1$, the allocation ${\delta }_i=\delta /2^i$ uses total budget $\delta$ by the geometric series. A fixed positive allowance for every requirement instead gives an infinite sum and no positive simultaneous-success guarantee from the union bound. The individual probability bounds must cover all the failure events being combined: omitting possible failure modes leaves the resulting guarantee about the listed requirements only. The same addition of error allowances underlies the Bonferroni correction in simultaneous hypothesis tests.

A filtered probability space is a probability space together with an increasing family of sigma-algebras ${\mathcal{F}}_t\subseteq \mathcal{F}$ describing the information available by time $t$.

On the full Cartesian product $\Omega ={\mathbb{R}}^{\mathbb{N}}$ with its product sigma-algebra, take the coordinate random variables $X_j(x)=x_j$. A product-measurable set $A$ is a tail event if and only if membership is unchanged whenever two sequences differ in finitely many coordinates. The measurability condition is essential: invariance alone is not a license to assign a probability to an arbitrary subset.

For the forward implication, fix $m$. An event in $\sigma (X_{m+1},X_{m+2},\dots )$ cannot distinguish points with the same remaining coordinates. Since a tail event belongs to this sigma-algebra for every $m$, every finite change preserves membership.

For the converse, freeze the first $m$ coordinates to zero. The insertion mapis a measurable function for the product sigma-algebras. Thus $B_m=i_m^{-1}(A)$ is a measurable set. Invariance gives $A={\pi }_m^{-1}(B_m)$, where the coordinate projection ${\pi }_m$ discards the first $m$ entries. Hence $A\in \sigma (X_{m+1},X_{m+2},\dots )$ for every $m$. The same argument works on $\{0,1{\}}^{\mathbb{N}}$. On a general probability space, coordinates need not admit arbitrary replacements, so the defining intersection of sigma-algebras remains the reliable formulation.

Let a probability space have countable measurable partitions of positive-measure cells, each partition refining its predecessor. An exceeding point has a first level where its cell average of $|f|$ exceeds $t$. The cells at those first crossings are disjoint: intersecting cells at different levels are nested, and an earlier exceeding cell prevents any of its descendants from being a first crossing. They cover exactly the maximal superlevel set. Summing their strict average inequalities gives the displayed weak type (1,1) estimate with constant one. The same proof works on a general measure space when every relevant cell has positive finite measure.

For three events in one classical probability space, the inclusion $A\setminus C\subset (A\setminus B)\cup (B\setminus C)$ and the union bound prove the displayed inequality. It is therefore an unconditional theorem of classical probability. In a polarization Bell test, a model assigning simultaneous setting-independent pass/fail outcomes to three possible analyzer directions must obey it. Equal unbiased marginals and quantum pair correlations instead give $P(A\setminus B)=\frac{1}{2}{\sin }^2({\theta }_A-{\theta }_B)$. Directions $0,\gamma ,2\gamma$ violate the resulting constraint for $0<\gamma <\pi /4$, because $2{\sin }^2\gamma <{\sin }^{2}2\gamma$. The conclusion excludes that common classical joint model. Conditional probabilities from sequential measurements cannot be inserted without their marginal factors or treated as if all three contexts shared an already established joint distribution.

For a finite-dimensional discounted gain random vector $X$, absence of arbitrage means $h\cdot X\ge 0$ almost surely only when $h\cdot X=0$ almost surely. Let $N=\{h:h\cdot X=0\text{ almost surely}\}$ and minimize $F(h)=\mathbb{E}{e}^{-|X{|}^2-h\cdot X}$ on the orthogonal complement $N^{\perp }$. The Gaussian damping makes $F$ finite and differentiable without physical integrability assumptions on $X$. Every nonzero direction in $N^{\perp }$ has negative gains on an event of positive probability; compactness of directions proves coercivity. At a minimizer, differentiation gives $\mathbb{E}[X{e}^{-|X{|}^2-h_{\ast }\cdot X}]=0$. Normalizing the positive weight produces an equivalent martingale measure with integrable gains. This is a constructive one-period proof of the fundamental theorem of asset pricing even on an infinite probability space.

In Information Theory and Statistics: A Tutorial by Imre Csiszár and Paul C. Shields (2004, Historical Notes, §3, printed p. 522), the authors connect systematic studies of information projection with earlier relative entropy minimization principles and geometric work, including Imre Csiszár's I-Divergence Geometry of Probability Distributions and Minimization Problems by Imre Csiszár (1975). This is the tutorial's historical account, rather than a claim of one isolated invention. Its finite-alphabet treatment makes the projection identities available without the additional machinery needed for general probability spaces.

An indicator random variable is the indicator function of an event $A$ in a probability space: $1_A$ is one when $A$ occurs and zero otherwise. It has a Bernoulli distribution with parameter $p=\mathbb{P}(A)$, so $\mathbb{E}[1_A]=p$ and $\mathrm{Var}(1_A)=p(1-p)$. Even an event described without numbers becomes a numerical random variable in this way.

For a finite collection of events, their indicator random variables add up to the number that occur. Linearity of expectation therefore turns an average count into a sum of probabilities, without requiring independence. The fixed point count of a uniform random permutation provides an example where the indicators are dependent random variables.