Probability space ($(\Omega ,\mathcal{F},\mathbb{P})$)

A probability space is a measure space $(\Omega ,\mathcal{F},\mathbb{P})$ whose total measure is one. The sample space $\Omega$ contains the elementary outcomes; the sigma-algebra $\mathcal{F}$ contains the events to which the model assigns probabilities; the probability measure $\mathbb{P}$ supplies those values. The sigma-algebra lets us combine observable questions using complements and countable unions while remaining within the model.

For two independent uniform dice, take $\Omega =\{1,\dots ,6{\}}^2$, $\mathcal{F}=\mathcal{P}(\Omega )$, and give each ordered pair mass $1/36$. A random variable might record the sum, the larger result, or whether both results are even integers. These are different functions on the same sample space, rather than different elementary experiments. A probability distribution records the mass of the values of one such random variable and can forget information retained by the full probability space.

For finitely many probability spaces $(X_i,{\mathcal{F}}_i,{\mu }_i)$, their product probability space has underlying set ${\prod }_i{X}_i$, the product sigma-algebra and product measure ${⨂}_i{\mu }_i$. The coordinate projections are jointly independent: for measurable $B_i\subseteq X_i$, the probability of all coordinate conditions is ${\prod }_i{\mu }_i(B_i)$. Conversely a family of independent coordinates has this product joint distribution. Product structure permits coordinate exposure and independent resampling in concentration arguments.

The sample space is the set $\Omega$ of outcomes represented by a probability model. One elementary outcome $\omega \in \Omega$ specifies a complete result at the level of detail the model keeps. An event is a subset belonging to the model's sigma-algebra $\mathcal{F}$; it occurs when the actual outcome lies in that subset. Together with a probability measure $\mathbb{P}$, these form a probability space $(\Omega ,\mathcal{F},\mathbb{P})$, with $\mathbb{P}(\Omega )=1$.^[ref]

Choosing a sample space means deciding which distinctions matter. For two tosses, recording the ordered head-or-tail sequence distinguishes $HT$ from $TH$; recording only the number of heads merges them. Neither representation is automatically wrong, but the second cannot answer a question about which toss was heads. Keeping a detailed sample space and defining a random variable for the desired observation lets several questions share one model.

A sample space alone assigns no probabilities. Even writing $\Omega =\{H,T{\}}^2$ as a Cartesian product does not establish independence: that requires a condition on the probability measure. On a finite sample space with all subsets admitted as events,Only when all elementary probabilities are equal does this become $\mathbb{P}(A)=|A|/|\Omega |$, where the bars denote cardinality. This is the uniform distribution on a finite set, an additional modelling choice.

Sample spaces can also be infinite. For example, take $\Omega =[0,1]$, its Borel sigma-algebra, and the continuous uniform distribution. Then $\mathbb{P}([a,b])=b-a$ for $0\le a\le b\le 1$, so each singleton has probability zero while $\mathbb{P}(\Omega )=1$. Thus an elementary outcome need not carry positive probability, and a continuous model cannot be understood by counting equally likely points. The distinction is developed in zero-probability event and countable additivity.

A family of disjoint events whose union is the sample space. For the countable law of total probability and law of total expectation, use a finite or countably infinite family. Conditioning formulas sum only positive-probability parts, or interpret zero-probability contributions as zero without assigning a conditional expectation there.

An elementary outcome is one member of the sample space. For the ordered two-toss model, $HT$ is one outcome; it contains both toss results. By contrast, “at least one head” is an event containing three outcomes. The outcome $HT$ and the singleton set $\{HT\}$ are different objects: when that singleton is measurable, the notation $\mathbb{P}(\{HT\})$ gives its probability. In the finite model below, every subset is measurable.

The word elementary refers to the chosen level of detail. An outcome can be an ordered pair, a sequence, or an entire realization of a point process. A random variable then extracts a numerical feature of that outcome; the window count of a point process is one example.

Model two fair independent coin tosses, recording their order. Here $H$ denotes heads and $T$ tails, andThe power set $\mathcal{P}(\Omega )$ includes every question expressible as a subset of these four outcomes. The mass $1/4=(1/2)(1/2)$ comes from the fairness and independence assumptions. These are exact model probabilities, rather than measured toss frequencies.

Let $A$ be the event that the first toss is heads and $B$ the event that the second toss is heads. ThenThe union means “at least one head,” while the intersection means “both heads.” Their probabilities are $3/4$ and $1/4$. Counting the union through the inclusion-exclusion principle gives $1/2+1/2-1/4=3/4$; alternatively its complement of an event is $\{TT\}$, of probability $1/4$.

On the same sample space, assigning masses $3/8,1/8,1/8,3/8$ to $HH,HT,TH,TT$ would still make each toss fair: $\mathbb{P}(A)=\mathbb{P}(B)=1/2$. But $\mathbb{P}(A\cap B)=3/8\ne 1/4$, so the tosses would be dependent. The event of at least one head would have probability $5/8$. Changing the probability measure changes the answer even when the sample space and the single-toss probabilities stay the same.

Define the head-count random variable $X:\Omega \to \{0,1,2\}$ byThe four elementary outcomes are inputs to this function; its three numerical values are outputs. Each fiber of a function collects all outcomes giving one value. Under the fair independent model, adding the masses in each fiber gives the probability mass functionConsequently the output values are not equally likely, although the inputs are. Using the reduced sample space $\{0,1,2\}$ with these masses gives the same head-count probability distribution; assigning $1/3$ to each value would describe a different model.

A question about a random variable's value becomes an event on the original sample space by taking an inverse image:For a real-valued random variable, this is an event whenever $D$ is a Borel set, by measurability. In this finite example every subset of $\{0,1,2\}$ is allowed. In particular,This is the same “at least one head” event counted above, expressed through an observable.^[ref]

Every event determined by $X$ is a union of complete fibers. There are $2^3=8$ such events, whereas the full power set of $\Omega$ has $2^4=16$. In particular, the first-head event $A=\{HH,HT\}$ is not determined by $X$: it contains $HT$ but excludes $TH$, although both give $X=1$. Choosing an observable makes calculations simpler precisely by forgetting distinctions; the useful choice is one that retains the distinctions the question needs.

An almost sure event is a measurable event of probability one. Countable intersections of such events still have probability one, allowing a conclusion to hold simultaneously for countably many parameter choices. Further deterministic reasoning may then extend it to uncountably many parameters, as in Brownian Hölder regularity or nowhere monotonicity of Brownian motion. An uncountable intersection need not preserve probability one without such additional reasoning.

A probability measure is a nonnegative, countably additive measure with total mass one. Thus $\mathbb{P}(\Omega )=1$, and for pairwise disjoint events $A_j$,The complement of an event has probability $1-\mathbb{P}(A)$. The probability of a union of overlapping events needs a correction: $\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cap B)$. Counting the overlap once is the simplest instance of the inclusion-exclusion principle.

A zero-probability event need not be impossible as a member of the sample space. Under the continuous uniform distribution on $[0,1]$, every individual point has probability zero, while the whole interval has probability one. Countable additivity applies to countable unions, not arbitrary uncountable ones; there is no contradiction.

Continuity of probability says that nested events can be approximated by their finite stages without losing probability in the limit of a sequence. In a probability space $(\Omega ,\mathcal{F},\mathbb{P})$, let $A_n,B_n\in \mathcal{F}$. ThenThe sigma-algebra ensures that both limiting events belong to $\mathcal{F}$. These are continuity from below of a measure and continuity from above of a measure. The first holds even for an infinite measure; the second requires finite measure of the first set, which is automatic for a probability measure because $\mathbb{P}(B_1)\le 1$.^[ref]

For a concrete model, toss a fair coin through a sequence of independent Bernoulli trials, and let $A_n$ be the event of at least one head in the first $n$ tosses. Its complement $B_n=A_n^c$ is the event of $n$ initial tails. On the sample space of infinite head-or-tail sequences,Every finite waiting horizon has positive tail probability, yet the event of waiting forever has probability zero. That intersection is not empty: it contains the all-tails sequence. Thus a zero-probability event need not be impossible as an outcome, and eventual success almost surely does not supply a deterministic deadline. The waiting time for the first head has a geometric distribution with success probability $1/2$. After ten tosses, the remaining tail probability is exactly $1/1024<1/1000$; nine tosses leave $1/512>1/1000$.

A proof outline exposes what makes countable additivity stronger than finite additivity of a set function. For increasing events, put $D_1=A_1$ and $D_n=A_n\setminus A_{n-1}$ for $n\ge 2$. These set differences are pairwise disjoint events, their union is $A$, and the first $n$ pieces have union $A_n$. ConsequentlyFor decreasing events, their complements increase to $B^c$; applying the increasing result and the complement rule of probability gives $\mathbb{P}(B_n)=1-\mathbb{P}(B_n^c)\to 1-\mathbb{P}(B^c)=\mathbb{P}(B)$. The finite-total-mass condition is essential for this subtraction. For comparison, the counting measure of each set $\{n,n+1,\dots \}$ of natural numbers is infinite, although their intersection is empty and has measure zero.

Conversely, on a sigma-algebra, a nonnegative normalized set function satisfying finite additivity of a set function and $R_n\downarrow \varnothing \Rightarrow \mathbb{P}(R_n)\to 0$ satisfies countable additivity. For pairwise disjoint events $D_j$ with $E={\bigcup }_j{D}_j$, the remainders $R_n=E\setminus {\bigcup }_{j=1}^n{D}_j$ decrease to the empty set, andTaking the limit of a sequence recovers the infinite sum. This equivalence explains why continuity of probability is part of the foundations rather than a separate assumption about smooth curves.

One useful application controls infinitely many possible errors. If events $E_j$ satisfy ${\sum }_j\mathbb{P}(E_j)<\infty$, then the events $C_n={\bigcup }_{j\ge n}{E}_j$ decrease, and the union bound gives $\mathbb{P}(C_n)\le {\sum }_{j\ge n}\mathbb{P}(E_j)\to 0$. Continuity of probability therefore makes $\mathbb{P}({\bigcap }_n{C}_n)=0$: errors occur infinitely often with probability zero. This is the Borel-Cantelli first lemma, and it requires no independence. For the illustrative bounds $\mathbb{P}(E_j)\le 2^{-j}$, the probability of any error at index $n$ or later is at most $2^{1-n}$, an explicit truncation guarantee.

The word continuity here concerns nested sets, not the absence of atoms in a probability distribution. For any real-valued random variable $X$, the events $\{X\le x+1/n\}$ decrease to $\{X\le x\}$. Hence its cumulative distribution function is right-continuous even when it has jumps, as for a geometric distribution. The nesting hypothesis matters: an arbitrary sequence of events need not admit either displayed interchange of limit and probability.

A probability measure is tight if compact sets carry arbitrarily nearly all its mass. Every Borel probability measure on a Polish space is tight. The stronger family condition is uniform tightness.

For càdlàg processes on a finite horizon, compact containment together with uniformly small increments over pairs of bounded stopping times whose separation tends to zero is a sufficient criterion for tightness in the Skorokhod J1 topology. The increment condition is in probability, not merely at deterministic times. If in addition the largest jump tends to zero in probability, every limiting law is supported on continuous paths.

The same compact set must work simultaneously for every measure in the family. Prokhorov's theorem relates this condition to relative compactness for the weak topology of probability measures on a Polish space.

A tight sequence of probability measures on the real line has a subsequence whose cumulative distribution functions converge at every continuity point of a cumulative distribution function. Choose a diagonal subsequence converging at every rational $q$, write those limits as $f(q)$, and define $F(x)={\inf }_{q>x}f(q)$. Monotonicity gives right continuity, and tightness of probability measures gives limits zero and one at the two ends of the real line. Bracketing by rational continuity approximations proves convergence at the continuity points of $F$. The corresponding probability measure is the weak subsequential limit: approximate a bounded continuous test function by step functions on a compact interval, and use tightness to bound the tails.

A zero-probability event has probability measure zero. It need not be empty: for a continuously distributed random variable, equality with a specified value is such an event. The complementary event occurs almost surely.

For observations or orbit points $x_0,\dots ,x_{N-1}$, the empirical measure is ${\nu }_N=N^{-1}{\sum }_{j=0}^{N-1}{\delta }_{x_j}$. Integrating a test function gives its sample average. For orbit points $x_j=T^{j}x$, the difference between $T_{\ast }{\nu }_N$ and ${\nu }_N$ telescopes to $({\delta }_{T^{N}x}-{\delta }_x)/N$, so on a compact metric space with continuous $T$, every weak limit is invariant.

If finite Borel measures $\mu$ and $\nu$ on $\mathbb{R}$ satisfyfor every compactly supported continuous function $f$, then $\mu =\nu$. This follows from the uniqueness part of the Riesz representation theorem; alternatively, continuous functions that approximate indicators from above and below recover the measure of every interval and hence every Borel set.

The Kolmogorov axioms describe a consistent way to assign probabilities to events. Start with a sample space $\Omega$ and an event sigma-algebra $\mathcal{F}$: it contains $\Omega$ and is closed under complements and countable unions, hence also countable intersections. The probability measure is a function $\mathbb{P}:\mathcal{F}\to \mathbb{R}$ satisfyingThe triple $(\Omega ,\mathcal{F},\mathbb{P})$ is a probability space. The third axiom is countable additivity; the infinite series means the limit of a sequence of nonnegative partial sums. It applies to pairwise disjoint events, not to overlapping events and not to arbitrary uncountable unions.^[ref]

The sigma-algebra specifies which questions have probabilities, while the axioms specify how those probabilities fit together. For a finite sample space one can take the entire power set; a continuous model commonly uses a Borel sigma-algebra. Nonnegativity and normalization do not require equally likely outcomes or independence: those are additional modelling choices.

For an exact six-outcome model, take $\Omega =\{1,2,3,4,5,6\}$, $\mathcal{F}=\mathcal{P}(\Omega )$, and assign $\mathbb{P}(\{k\})=k/21$. The total is one because $1+2+3+4+5+6=21$. Define the probability of an event by summing the weights of its outcomes. The disjoint events $A=\{1,2\}$, $B=\{3,4\}$ and $C=\{5,6\}$ form a partition of a set withThis is a specified mathematical model, not measured die frequencies. The figure makes probability literal area: each region has the same height and a width proportional to its assigned mass. Regrouping disjoint events preserves their total area.

Several familiar rules are consequences, not further axioms. Finite additivity of a set function follows by appending empty events to the sequence in countable additivity; $\mathbb{P}(\varnothing )=0$ follows already by writing $\Omega =\Omega \cup \varnothing$ as a disjoint union. The complement rule of probability gives $\mathbb{P}(A^c)=1-\mathbb{P}(A)$ and therefore $0\le \mathbb{P}(A)\le 1$. Splitting larger events into a smaller event and a set difference shows $A\subseteq B\Rightarrow \mathbb{P}(A)\le \mathbb{P}(B)$.

For overlapping events, the inclusion-exclusion principle corrects double counting:In the six-outcome model, $D=\{2,3\}$ and $E=\{3,4\}$ give $5/21+7/21-3/21=9/21$, agreeing with the weight of $\{2,3,4\}$. This decomposition is useful because a complicated event can be assembled from simpler questions with known probabilities.

On a finite sample space, a family of pairwise disjoint events has only finitely many nonempty members, so finite additivity of a set function already handles every countable disjoint union. On an infinite sample space, the countable axiom additionally controls what is left beyond every finite stage. In the fair-coin waiting model of continuity of probability, the finite-stage calculations $\mathbb{P}(\text{first head at toss }j)=2^{-j}$ sum to $1-2^{-n}$ through toss $n$. Countable additivity makes the probability of a head at some finite toss equal to ${\sum }_{j\ge 1}{2}^{-j}=1$; finite-stage sums alone do not justify that passage. For the continuous uniform distribution on $[0,1]$, each singleton instead has probability zero, while their uncountable union has probability one. Countable, rather than unrestricted, additivity accommodates both models.

Historically, Kolmogorov's 1933 book Grundbegriffe der Wahrscheinlichkeitsrechnung established the influential axiomatic framework connecting probability theory with measure theory. The three conditions above are the modern presentation on a sigma-algebra. Shafer and Vovk's historical study explains that the original presentation began with finite additivity on an algebra of sets and added continuity for decreasing events with empty intersection, which supplies the countable extension.^[ref]

Let $(B_i)$ be a countable measurable partition of the sample space, with positive probabilities. For any event $A$, the law of total probability givesThe pieces $A\cap B_i$ are disjoint events, so add their masses and use the definition of conditional probability. Zero-mass partition pieces can be omitted without changing the answer.

For the coin selection in probability, let $B$ mean the $3/4$-head coin was selected and let $H$ mean the first toss was heads. Then $\mathbb{P}(H)=\mathbb{P}(H\mid B)\mathbb{P}(B)+\mathbb{P}(H\mid B^c)\mathbb{P}(B^c)=11/20$. Each conditional answer describes one possible hidden situation; its weight describes how often that situation occurs under the model. This decomposition is useful whenever a simple conditional model sits behind a more complicated marginal probability distribution.

The law of total probability combines local conditional probabilities into one overall probability. The measurable partition must cover the sample space with pairwise disjoint events: every elementary outcome belongs to exactly one part. Its probabilities are the weights of a weighted mean, and they sum to one.^[ref]

For an original constructed example, make a finite set of 1,000 items: 600 belong to $B_1$ and 400 to $B_2$. Mark exactly 60 items in $B_1$ and 100 in $B_2$, and let the event $A$ mean that the selected item is marked. Select an item using the uniform distribution on a finite set. These are prescribed counts, so the resulting probabilities are exact properties of this construction:The conditional probability $0.1$ uses only the 600 items in $B_1$ as its denominator. The probability of the intersection $A\cap B_1$ uses all 1,000 items instead: $0.6\times 0.1=0.06$. Similarly, $\mathbb{P}(A\cap B_2)=0.4\times 0.25=0.10$. Adding the two disjoint events gives

An unweighted arithmetic mean of the two conditional probabilities would give $(0.1+0.25)/2=0.175$. That is the answer to a different selection rule: choose either group with probability $1/2$, then select uniformly within that group. Here the first group contains more items, so selecting uniformly among all items gives it weight $0.6$. In fact, if its weight were $w$, the overall probability would be $0.1w+0.25(1-w)=0.25-0.15w$. Moving the weight from $1/2$ to $0.6$ lowers the result by $0.015$, explaining exactly why $0.175$ is too large for the original model.

The same construction is a mixture distribution: first select a group with its mixture weight, then select an item within that group. Multiplying the weight by its conditional probability gives the probability of reaching $A$ through that group; adding the contributions gives the law of total probability. For example, an item in $B_1$ has selection probability $0.6/600=1/1000$, and an item in $B_2$ has selection probability $0.4/400=1/1000$, recovering the original uniform distribution on a finite set.

No assumption of independent events is needed. In this construction $\mathbb{P}(A\mid B_1)=0.1$ differs from $\mathbb{P}(A)=0.16$, so $A$ and $B_1$ are not independent events. Also $B_1$ and $B_2$ are disjoint events with positive probabilities, so they cannot be independent events: their intersection has probability zero, whereas $\mathbb{P}(B_1)\mathbb{P}(B_2)=0.24$. The measurable partition supplies exclusive alternatives, not an independence assumption.

For a short proof, the intersection pieces $A\cap B_i$ have union $A$ and are pairwise disjoint events. Apply countable additivity, then substitute the definition of conditional probability on every positive-mass part:A zero-probability event $B_i$ has $\mathbb{P}(A\cap B_i)=0$. Omit it: writing $\mathbb{P}(A\mid B_i)$ would divide by zero. Countability ensures that the union of the omitted zero-probability events also has probability zero. Overlapping parts would double-count their intersections, and incomplete coverage would omit the probability of $A$ in the uncovered region.^[ref]

Reversing the question gives one compact application of Bayes' theorem. Among the 160 marked items, 100 belong to $B_2$, so its conditional probability after observing $A$ isThe law of total probability supplied the denominator by collecting every way that $A$ can occur. Its usefulness lies in choosing a measurable partition where the conditional probabilities are simple, then combining them without losing the relative sizes of the alternatives.

Let a discrete random variable $Y$ have probability mass function $p_y$. Given $Y=y$, return $y$ with probability $a_y\in [0,1]$; otherwise draw from a conditional distribution with probability mass function $K(y,x)$. The law of total probability gives the displayed identity. Its total mass is ${\sum }_y{p}_y[a_y+(1-a_y){\sum }_{x}K(y,x)]=1$. If the fallback is a discrete uniform distribution on $0,\dots ,y$, then $K(y,x)=1_{\{0\le x\le y\}}/(y+1)$, so the fallback sum includes $y=x$.

Write $q=P(A)$, $r=P(B\mid A)$ and $s=P(B\mid A^c)$. The law of total probability gives the displayed convex combination. With $q,r$ fixed and $s$ free in $[0,1]$,With $r,s$ fixed and $q$ free in $[0,1]$,The joint cell masses $qr,q(1-r),(1-q)s,(1-q)(1-s)$ are nonnegative and sum to one, proving sharpness in the joint-model interpretation. For ordinary conditional probabilities, a conditioning event must have positive mass. If both $r$ and $s$ are specified by ratios, then $0<q<1$: for $r\ne s$ the last bounds are strict and their endpoints are the infimum and supremum. If $r=s$, the marginal equals that common value for every $q$. Endpoint mixtures remain meaningful if the unused zero-mass conditional is treated as a prescribed conditional model rather than a defined ratio.

An event is a measurable set in a probability space: a question with a yes-or-no answer for each outcome. On $\Omega =\{1,\dots ,6{\}}^2$, the event that the sum is at least nine consists of ten ordered pairs. Its probability is therefore $10/36=5/18$ under independent uniform dice. Saying that an event occurred means the actual outcome belongs to that set.

For events $A_n$ in a probability space, the infinitely-often event isIts complement of an event is ${\bigcup }_{m\ge 1}{\bigcap }_{n\ge m}{A}_n^c$: after some outcome-dependent index, no more events occur.^[ref]

Deleting finitely many events leaves the limsup of events unchanged. It is therefore a tail event of the sequence of indicator random variables $1_{A_n}$, whether or not those random variables are independent. For independent events, the Kolmogorov zero-one law restricts its probability to zero or one; the Borel-Cantelli lemmas determine which alternative holds from the series of probabilities. Also $1_{A_n}\to 0$ precisely when only finitely many events occur, connecting the First Borel-Cantelli lemma to almost sure convergence and the existing complete convergence criterion.

Let $(a_n)$ be a nondecreasing real sequence tending to $+\infty$, and let $(x_n)$ be any real sequence. The running maximum exceeds $a_n$ infinitely often exactly when $x_n>a_n$ infinitely often. Applying this pathwise to random variables gives the displayed event equality, without any independence assumption.

One implication follows from $x_n\le {\max }_{k\le n}{x}_k$. For the other, if $x_n\le a_n$ from some $N$ onwards, every later value with $N\le k\le n$ satisfies $x_k\le a_k\le a_n$. The finite prefix also lies below $a_n$ for all sufficiently large $n$, because $a_n\to \infty$. Therefore the entire running maximum eventually stays below the threshold.

Divergence of the thresholds prevents one large early observation from creating infinitely many crossings. Monotonicity ensures that a later running-maximum crossing is controlled by the corresponding earlier individual threshold. These two conditions explain why the resulting event belongs to the tail sigma-algebra when the thresholds are deterministic.

For any sequence of events, its limsup of events satisfiesSet $C_N={\bigcup }_{n\ge N}{A}_n$. These are decreasing events with intersection ${\mathrm{limsup}}_n{A}_n$. Continuity of probability from above givesThis is also the bounded-indicator form of the reversed Fatou lemma. The inequality can be strict: independent fair-coin head events each have probability $1/2$, while their limsup of events has probability $1$ by the Second Borel-Cantelli lemma.

The event that $A$ does not occur has probability $1-P(A)$. Conditioning on an event positively associated with $A$ negatively associates its complement with $A$.

Two events are disjoint when they cannot both occur. For a finite family of pairwise disjoint events, the probability of their union is the sum of their probabilities, so that sum is at most one. This condition differs from the condition defining independent events.

On a product space ${\mathbb{R}}^T$, the cylinder sigma-algebra is generated by coordinate projections. Every event in it depends on at most countably many coordinates.