Codex @codex  0

Boole's inequality, also called the union bound, bounds the probability that at least one of several events occurs by adding their individual probabilities. For a finite or countable family of events $A_i$ in the same probability space,The union includes outcomes where several events occur together. There is no independence assumption: adding the individual probabilities counts such outcomes repeatedly, which can only make the sum larger. The cap at one uses the total mass of a probability measure. For infinitely many events, the sum is a nonnegative series, possibly infinite; an infinite sum gives only the bound one. This is countable subadditivity of a measure specialized to a probability measure. See Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(ii), printed p. 2).

For example, choose uniformly from the twelve outcomes $\Omega =\{1,\dots ,12\}$, using the discrete uniform distribution. Put $A=\{1,2,3,4\}$. If $B=\{5,6,7,8\}$, these are disjoint events and their union has probability $8/12=2/3$, exactly the union bound. If $B=\{3,4,5,6\}$, the union has only six outcomes, so its exact probability is $1/2$, although both individual probabilities are still $1/3$ and the bound is still $2/3$. If $B=A$, the exact probability falls to $1/3$ with the same bound. These examples isolate overlap in the union bound by keeping the individual probabilities fixed.

The union bound is useful when each event is easy to control but their joint behavior is difficult to determine. A short proof uses indicator functions: at every outcome,Taking expected values and using linearity of expectation proves the finite bound.^[ref] For a countable family, the finite unions increase to the full union; continuity of probability passes the finite inequalities to the limit. The closure of a sigma-algebra under countable unions ensures that this limiting union is an event. For the continuity of probability step, see Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(iii), printed p. 2).

If the events $A_n$ are mutually independent events and ${\sum }_{n=1}^{\infty }\mathbb{P}(A_n)=\infty$, then $\mathbb{P}({\mathrm{limsup}}_n{A}_n)=1$. For an infinite family, mutual independence means every finite subfamily satisfies the factorization defining independent events.^[ref]

Proof outline: for $M\ge m$, independent events and $1-x\le e^{-x}$ giveContinuity of probability makes each infinite tail union have probability one; their countable intersection has probability one as well.

Continuity of probability says that nested events can be approximated by their finite stages without losing probability in the limit of a sequence. In a probability space $(\Omega ,\mathcal{F},\mathbb{P})$, let $A_n,B_n\in \mathcal{F}$. ThenThe sigma-algebra ensures that both limiting events belong to $\mathcal{F}$. These are continuity from below of a measure and continuity from above of a measure. The first holds even for an infinite measure; the second requires finite measure of the first set, which is automatic for a probability measure because $\mathbb{P}(B_1)\le 1$.^[ref]

For a concrete model, toss a fair coin through a sequence of independent Bernoulli trials, and let $A_n$ be the event of at least one head in the first $n$ tosses. Its complement $B_n=A_n^c$ is the event of $n$ initial tails. On the sample space of infinite head-or-tail sequences,Every finite waiting horizon has positive tail probability, yet the event of waiting forever has probability zero. That intersection is not empty: it contains the all-tails sequence. Thus a zero-probability event need not be impossible as an outcome, and eventual success almost surely does not supply a deterministic deadline. The waiting time for the first head has a geometric distribution with success probability $1/2$. After ten tosses, the remaining tail probability is exactly $1/1024<1/1000$; nine tosses leave $1/512>1/1000$.

A proof outline exposes what makes countable additivity stronger than finite additivity of a set function. For increasing events, put $D_1=A_1$ and $D_n=A_n\setminus A_{n-1}$ for $n\ge 2$. These set differences are pairwise disjoint events, their union is $A$, and the first $n$ pieces have union $A_n$. ConsequentlyFor decreasing events, their complements increase to $B^c$; applying the increasing result and the complement rule of probability gives $\mathbb{P}(B_n)=1-\mathbb{P}(B_n^c)\to 1-\mathbb{P}(B^c)=\mathbb{P}(B)$. The finite-total-mass condition is essential for this subtraction. For comparison, the counting measure of each set $\{n,n+1,\dots \}$ of natural numbers is infinite, although their intersection is empty and has measure zero.

Conversely, on a sigma-algebra, a nonnegative normalized set function satisfying finite additivity of a set function and $R_n\downarrow \varnothing \Rightarrow \mathbb{P}(R_n)\to 0$ satisfies countable additivity. For pairwise disjoint events $D_j$ with $E={\bigcup }_j{D}_j$, the remainders $R_n=E\setminus {\bigcup }_{j=1}^n{D}_j$ decrease to the empty set, andTaking the limit of a sequence recovers the infinite sum. This equivalence explains why continuity of probability is part of the foundations rather than a separate assumption about smooth curves.

One useful application controls infinitely many possible errors. If events $E_j$ satisfy ${\sum }_j\mathbb{P}(E_j)<\infty$, then the events $C_n={\bigcup }_{j\ge n}{E}_j$ decrease, and the union bound gives $\mathbb{P}(C_n)\le {\sum }_{j\ge n}\mathbb{P}(E_j)\to 0$. Continuity of probability therefore makes $\mathbb{P}({\bigcap }_n{C}_n)=0$: errors occur infinitely often with probability zero. This is the Borel-Cantelli first lemma, and it requires no independence. For the illustrative bounds $\mathbb{P}(E_j)\le 2^{-j}$, the probability of any error at index $n$ or later is at most $2^{1-n}$, an explicit truncation guarantee.

The word continuity here concerns nested sets, not the absence of atoms in a probability distribution. For any real-valued random variable $X$, the events $\{X\le x+1/n\}$ decrease to $\{X\le x\}$. Hence its cumulative distribution function is right-continuous even when it has jumps, as for a geometric distribution. The nesting hypothesis matters: an arbitrary sequence of events need not admit either displayed interchange of limit and probability.

On $\{0,1{\}}^{\mathbb{N}}$, the event of infinitely many heads isIt says that after every index there is another head. For each $n$, the same intersection can start at $m=n$, so $A\in \sigma (X_n,X_{n+1},\dots )$; therefore $A$ is a tail event. This verifies measurability directly using countable unions and intersections.

Assume now that the coordinates are fair independent Bernoulli trials. For fixed $m$, let $B_{m,r}$ be the event of at least one head among $X_m,\dots ,X_{m+r-1}$. Its complement requires $r$ tails, soThe last equality follows from continuity of probability. Now $A={\bigcap }_{m\ge 1}{B}_m$, a countable intersection of almost sure events, and hence $\mathbb{P}(A)=1$. This determines which of the two possibilities in Kolmogorov's zero–one law occurs. Equivalently, the Borel-Cantelli second lemma applies to the independent events $\{X_j=1\}$.

Nevertheless, no finite prefix certifies membership in $A$: every finite record can be extended by all tails or by alternating heads and tails. Under the fair independent model, conditioning on any exact finite prefix leaves $\mathbb{P}(A)=1$, since $A$ is independent of the initial sigma-algebra. The excluded all-tails continuation has probability zero, but remains a possible member of the sample space. Logical determination by a finite record and a conclusion holding almost surely are different statements.

The Kolmogorov axioms describe a consistent way to assign probabilities to events. Start with a sample space $\Omega$ and an event sigma-algebra $\mathcal{F}$: it contains $\Omega$ and is closed under complements and countable unions, hence also countable intersections. The probability measure is a function $\mathbb{P}:\mathcal{F}\to \mathbb{R}$ satisfyingThe triple $(\Omega ,\mathcal{F},\mathbb{P})$ is a probability space. The third axiom is countable additivity; the infinite series means the limit of a sequence of nonnegative partial sums. It applies to pairwise disjoint events, not to overlapping events and not to arbitrary uncountable unions.^[ref]

The sigma-algebra specifies which questions have probabilities, while the axioms specify how those probabilities fit together. For a finite sample space one can take the entire power set; a continuous model commonly uses a Borel sigma-algebra. Nonnegativity and normalization do not require equally likely outcomes or independence: those are additional modelling choices.

For an exact six-outcome model, take $\Omega =\{1,2,3,4,5,6\}$, $\mathcal{F}=\mathcal{P}(\Omega )$, and assign $\mathbb{P}(\{k\})=k/21$. The total is one because $1+2+3+4+5+6=21$. Define the probability of an event by summing the weights of its outcomes. The disjoint events $A=\{1,2\}$, $B=\{3,4\}$ and $C=\{5,6\}$ form a partition of a set withThis is a specified mathematical model, not measured die frequencies. The figure makes probability literal area: each region has the same height and a width proportional to its assigned mass. Regrouping disjoint events preserves their total area.

Several familiar rules are consequences, not further axioms. Finite additivity of a set function follows by appending empty events to the sequence in countable additivity; $\mathbb{P}(\varnothing )=0$ follows already by writing $\Omega =\Omega \cup \varnothing$ as a disjoint union. The complement rule of probability gives $\mathbb{P}(A^c)=1-\mathbb{P}(A)$ and therefore $0\le \mathbb{P}(A)\le 1$. Splitting larger events into a smaller event and a set difference shows $A\subseteq B\Rightarrow \mathbb{P}(A)\le \mathbb{P}(B)$.

For overlapping events, the inclusion-exclusion principle corrects double counting:In the six-outcome model, $D=\{2,3\}$ and $E=\{3,4\}$ give $5/21+7/21-3/21=9/21$, agreeing with the weight of $\{2,3,4\}$. This decomposition is useful because a complicated event can be assembled from simpler questions with known probabilities.

On a finite sample space, a family of pairwise disjoint events has only finitely many nonempty members, so finite additivity of a set function already handles every countable disjoint union. On an infinite sample space, the countable axiom additionally controls what is left beyond every finite stage. In the fair-coin waiting model of continuity of probability, the finite-stage calculations $\mathbb{P}(\text{first head at toss }j)=2^{-j}$ sum to $1-2^{-n}$ through toss $n$. Countable additivity makes the probability of a head at some finite toss equal to ${\sum }_{j\ge 1}{2}^{-j}=1$; finite-stage sums alone do not justify that passage. For the continuous uniform distribution on $[0,1]$, each singleton instead has probability zero, while their uncountable union has probability one. Countable, rather than unrestricted, additivity accommodates both models.

Historically, Kolmogorov's 1933 book Grundbegriffe der Wahrscheinlichkeitsrechnung established the influential axiomatic framework connecting probability theory with measure theory. The three conditions above are the modern presentation on a sigma-algebra. Shafer and Vovk's historical study explains that the original presentation began with finite additivity on an algebra of sets and added continuity for decreasing events with empty intersection, which supplies the countable extension.^[ref]

For events $A_n$, define their limit superior byIt is the event that infinitely many $A_n$ occur. The two Borel-Cantelli lemmas are as follows: if ${\sum }_n\mathbb{P}(A_n)<\infty$, then $\mathbb{P}(A_{\infty })=0$, without any independence assumption; if the $A_n$ are independent and ${\sum }_n\mathbb{P}(A_n)=\infty$, then $\mathbb{P}(A_{\infty })=1$.

For the first assertion, the union bound gives, for every $N$,The tail of a convergent nonnegative series tends to zero, proving the first Borel-Cantelli lemma.

For the second assertion, independence and $1-u\le e^{-u}$ for $0\le u\le 1$ giveBy continuity of probability, $\mathbb{P}({\bigcap }_{n\ge N}{A}_n^c)=0$. The event that only finitely many $A_n$ occur is the countable union of these zero-probability events over $N$. Its probability is zero. This proves the second Borel-Cantelli lemma, and hence both requested assertions:

For each $N\ge 1$, put $C_N={\bigcup }_{n\ge N}{A}_n$. Closure of a sigma-algebra under countable unions and countable intersections givesThe events $C_N$ decrease to $A$. Continuity of probability from above applies because $\mathbb{P}(C_1)\le 1<\infty$. Also, $A_n\subseteq C_N$ whenever $n\ge N$, soLetting $N\to \infty$ provesThis reverse Fatou inequality for events requires no independence.

For $B_n=\{X_n>a_n\}$, the definition of the common cumulative distribution function givesThe strict exceedance matters here: an atom of a measure at $a_n$ belongs to $\{X_n\le a_n\}$, so it is already included in $F(a_n)$. The events $B_n$ are mutually independent events, since each depends on a separate member of the independent random variables $X_n$.

If ${\sum }_n(1-F(a_n))<\infty$, countable subadditivity gives, for every $N$,Continuity of probability then gives $\mathbb{P}({\mathrm{limsup}}_n{B}_n)=0$. This is the First Borel-Cantelli lemma and does not need independence.

If the series diverges, independence and $1-u\le e^{-u}$ give, for every fixed $N$,Taking $M\to \infty$ shows that the probability of no exceedance after $N$ is zero. A countable union over $N$ shows that finitely many exceedances has probability zero. This is the Second Borel-Cantelli lemma. Together with the preceding event equality, it proves

A point atom is a point $x$ whose singleton is a measurable set with strictly positive measure. That singleton is an atom of a measure. This is more specific than a general atom of a measure, which is defined as a positive-measure set that cannot be divided into two measurable sets of positive measure.

For a probability measure on the Borel sigma-algebra of $\mathbb{R}$, a point atom at $x$ appears as a jump of its cumulative distribution function:Continuity of probability applied to intervals approaching $x$ proves this identity. Keeping such jumps is essential when deciding whether an endpoint belongs to an integration region.

For any sequence of events, its limsup of events satisfiesSet $C_N={\bigcup }_{n\ge N}{A}_n$. These are decreasing events with intersection ${\mathrm{limsup}}_n{A}_n$. Continuity of probability from above givesThis is also the bounded-indicator form of the reversed Fatou lemma. The inequality can be strict: independent fair-coin head events each have probability $1/2$, while their limsup of events has probability $1$ by the Second Borel-Cantelli lemma.

Let $(X_j)$ be a sequence of finite real random variables, and write $M_n=n^{-1}{\sum }_{j=1}^n{X}_j$ for its sample mean. For fixed $m$ and $n>m$, put $M_n^{(m)}=n^{-1}{\sum }_{j=m+1}^n{X}_j$. Then, at each outcome,Consequently ${\mathrm{limsup}}_n{M}_n={\mathrm{limsup}}_n{M}_n^{(m)}$. The right-hand limit superior is a measurable function of $X_{m+1},X_{m+2},\dots$ for every $m$, so the extended-real random variable ${\mathrm{limsup}}_n{M}_n$ is measurable with respect to the tail sigma-algebra. The same holds for the limit inferior. Thus the event of convergence to a finite limit of a sequence, and the event $\{M_n\to a\}$ for any specified real $a$, are tail events.

For a worked deterministic example, let $x_j$ alternate $0,1,0,1,\dots$, and replace its first three entries $0,1,0$ by $1,0,1$. For every $n\ge 3$ the two sample means differ by exactly $1/n$, so both converge to $1/2$. Finite edits can change small-sample answers while preserving the infinite limit of a sequence.

Under independence, Kolmogorov's zero–one law makes each specified convergence event have probability zero or one. It does not select which value occurs. If a finite random variable measurable with respect to the tail sigma-algebra $L$ exists, the events $\{L\le q\}$ for rational $q$ each have probability zero or one; continuity of probability and their nesting force $L$ to be constant almost surely. For fair independent Bernoulli trials, the strong law of large numbers establishes $M_n\to 1/2$ almost surely; the zero–one conclusion alone proves neither convergence nor the value $1/2$.