Sample space Created 2026-10-14 Updated 2026-10-14
The sample space is the set of outcomes represented by a probability model. One elementary outcome specifies a complete result at the level of detail the model keeps. An event is a subset belonging to the model's sigma-algebra ; it occurs when the actual outcome lies in that subset. Together with a probability measure , these form a probability space , with .[ref]
Choosing a sample space means deciding which distinctions matter. For two tosses, recording the ordered head-or-tail sequence distinguishes from ; recording only the number of heads merges them. Neither representation is automatically wrong, but the second cannot answer a question about which toss was heads. Keeping a detailed sample space and defining a random variable for the desired observation lets several questions share one model.
A sample space alone assigns no probabilities. Even writing as a Cartesian product does not establish independence: that requires a condition on the probability measure. On a finite sample space with all subsets admitted as events,
Only when all elementary probabilities are equal does this become , where the bars denote cardinality. This is the uniform distribution on a finite set, an additional modelling choice.
Sample spaces can also be infinite. For example, take , its Borel sigma-algebra, and the continuous uniform distribution. Then for , so each singleton has probability zero while . Thus an elementary outcome need not carry positive probability, and a continuous model cannot be understood by counting equally likely points. The distinction is developed in zero-probability event and countable additivity.
For a nonnegative random variable with positive expected value and finite second moment, write . The Cauchy-Schwarz inequality gives
which proves the bound in second moment method. It is the zero-threshold case of the Paley-Zygmund inequality. Equality occurs when is a fixed positive value on the event and zero elsewhere, almost surely. For with a Bernoulli random variable of parameter , equals the exact positivity probability.
The variance identity turns the lower bound into a statement about variance relative to the squared mean. A small ratio forces positivity to be likely. A positive expected value alone does not do so: keeping while taking with probability and zero otherwise makes the positivity probability as small as any chosen .
Serial reaction dwell-time density Created 2026-10-25 Updated 2026-11-05
Two sequential independent memoryless reaction waits have a hypoexponential distribution. convolution gives . Equal rates give the Erlang distribution . A nonzero mode distinguishes this ideal two-event model from a single exponential wait, but does not chemically identify the events.
Choose one random variable with the continuous uniform distribution on , and reuse it in every event:
Each probability is , so their harmonic series diverges, just as in summable and harmonic event probabilities. But the events are decreasing: . For every outcome , all indices with fail. Only finitely many events occur on that outcome. The limsup of events is exactly , a zero-probability event. This is a concrete closed-interval version of the shrinking-interval dependence example.[ref]
The missing assumption is visible already in two events. For , their intersection is , hence
Moreover, the whole tail union is just , with probability . This calculation explains directly why divergence alone fails: overlapping events keep spending their individual probabilities on the same outcomes.
Let be the total number of occurrences, a sum of indicator random variables. Although almost surely, Tonelli theorem gives . More explicitly, for : very small positive values of give very long, but finite, initial runs. Infinite expected value does not imply a positive probability of an infinite count.
Figure 1.
Equal harmonic event probabilities with fresh independent inputs or one shared input
. Each row is one finite simulation of the indicator random variables of for ; a dot means the event occurred. Fresh independent random variables allow scattered returns; a shared uniform random variable yields an initial run followed by zeros. The horizontal axis uses a logarithmic scale to show early runs clearly. These finite simulations illustrate the two models and cannot prove infinitely many occurrences or eventual stopping. Details.
Let a random vector have a finite actual range of values. Its nonempty fibres partition the sample space, and every event in is a union of those fibres. Thus each fibre is an atom of a sigma-algebra and has exactly elements. This assertion concerns the actual range; additional values attained only on null sets must still be counted when working with an uncompleted sigma-algebra literally.
To verify the claim, the fibres belong to as preimages of singleton Borel sets. Their unions form a finite sigma-algebra making measurable, so minimality of the generated sigma-algebra gives the reverse inclusion. Distinct subcollections have distinct unions because the fibres are nonempty and disjoint.
Consequently a finite history of finite-valued observations generates a finite natural filtration at each fixed time. It may be strictly smaller than the filtration of its driving continuous observations: a single uniform random variable already generates infinitely many distinct events with different probabilities.
Fix a failure probability and assume an independent and identically distributed statistical sample. A finite-sample confidence band for its population cumulative distribution function is
Here is the natural logarithm. The DKW inequality implies
This is simultaneous coverage over the whole real line, valid at each fixed sample size.[ref]
To obtain the band, solve for the positive . Outside a failure event of probability at most , for every . Since a cumulative distribution function already lies in , clipping the endpoints to this interval preserves the coverage event. The guarantee is at least , and need not equal it.
Choose , giving at least simultaneous coverage under the stated model. For , respectively, the half-widths before clipping are
For the chosen five observations, the band at is , while at it is approximately . At the unclipped band would be , so clipping gives . The broad uncertainty is informative: five observations give a precise description of themselves, but a coarse bound for an unknown population cumulative distribution function.
Figure 1.
Finite-sample simultaneous uncertainty around an empirical distribution function
. Left: the chosen five observations give and its band clipped to , using . Right: theoretical half-widths for , with no additional samples constructed. The confidence band has coverage at least under independent and identically distributed sampling; the drawing illustrates the procedure and does not establish its coverage. Details.
A collection of pointwise confidence intervals only promises coverage separately at each preselected threshold; it does not promise coverage of an entire cumulative distribution function. Here the one simultaneous event permits inspecting any threshold after seeing the data. The probability refers to repeated sampling of the entire statistical sample, with fixed. It does not assign probability to a fixed after observing this particular band, and it is not a prediction interval for a future observation. The displayed comparison gives a guarantee for each fixed separately, not simultaneous validity over all sample sizes or an optional stopping rule. The chosen observations are an illustration; the coverage claim follows from the DKW inequality, not from their appearance or a simulation. Dependence or observations with different sampling laws require another argument.
At fixed , multiplying by divides the half-width by . Conversely, a desired half-width is ensured by , rounded up to an integer. For and , this sufficient sample size is . This makes the confidence band useful for planning how many observations are needed before estimating threshold probabilities to a chosen simultaneous accuracy.
Square of a standard Laplace variable Created 2026-10-25 Updated 2026-11-05
For a random variable with probability density function , the event is for . Integrating the symmetric density gives the displayed cumulative distribution function; it is zero for negative . The density on is . Its integrable singularity at zero does not create an atom, because the cumulative distribution function is continuous there.
Standardization of a normal random variable Created 2026-09-24 Updated 2026-10-14
If with , then has the standard normal distribution. The change-of-variables formula for a probability density gives , the standard normal density. Thus standardization of a normal random variable removes both parameters without changing the probabilities of corresponding events.
Stationary jump traffic at a single contact Created 2026-10-25 Updated 2026-11-05
Use the master-equation convention that is the transition rate from to . In a stationary law , the mean directed transition count per unit time is
The departure-state occupation multiplies the departure-to-destination rate. The reversed rate index would describe a different transition. A physical event transporting signed charge in the forward direction has mean charge current .[ref]
For a closed finite chain whose complete transition graph is a tree, every stationary edge current is zero. Removing one edge separates the states into two sets; stationarity makes total probability flow across that cut vanish, and the removed edge is its only crossing. Both directed counts can nevertheless be positive. On a three-state path with symmetric adjacent rates , the stationary law is uniform and the two edge traffic rates are and in each direction.
Different physical transition mechanisms between the same states should be kept as separate edges of a multigraph when their currents matter. Summing their rates produces an aggregated edge. Opposing mechanism currents can cancel in that aggregate, so zero aggregate current does not force every mechanism's current to vanish.
Stochasticity Created 2026-10-20 Updated 2026-11-05
Random variation represented by a probability distribution or stochastic process, whether from intrinsic events or an uncertain environment.
The same stopping rule connects a waiting count to a fixed-length count. Let be the trial number of the th success, and let count successes in the first trials, so has a binomial distribution. For any nonnegative integer ,
The target is still unmet after trials exactly when fewer than successes have occurred. With , and , this gives ; equivalently, , matching . At , the event identity still holds, with and deterministically.
Success-failure stopping identity Created 2026-10-18 Updated 2026-10-20
In independent Bernoulli trials with success probability and failure probability , let be the trial of success and the trial of failure , for positive integers . If success occurs after failures, its final trial is a success and the preceding trials have successes and failures. The negative binomial stopping argument therefore gives the title sum.
Put . By trial , at least one target must have been reached, so the same event is that a binomial distribution with trials has at least successes. Conditioning instead on the final trial of a block of trials gives
Indeed, more than successes in the full block guarantee the success target won; with exactly successes it won only when the final trial was a failure. These disjoint cases explain the extra boundary term. A zero success target wins immediately; a zero failure target has already been reached. To include in the displayed finite sum requires the explicit generalized convention ; an upper summation limit of denotes an empty sum.
Take independent random variables , each with the continuous uniform distribution on , and set . Then , and the events are mutually independent events. The product probability space supplies a model for the entire infinite sequence; a finite list of simulated values is only an illustration.
For , the p-series is summable:
The Basel problem supplies the total sum, and the integral test for convergence gives the displayed tail bound. The First Borel-Cantelli lemma makes the limsup of events have probability zero. In fact, the tail bound implies that the probability of even one alarm at or after is at most ; for , it is at most . This bound remains valid for dependent events with the same individual probabilities.
For , the harmonic series diverges, so the Second Borel-Cantelli lemma makes the limsup of events have probability one. There is also an exact finite-horizon calculation using independent events:
For any fixed , this finite product of numbers tends to zero as . Every infinite tail therefore contains an alarm almost surely, even though the individual probabilities tend to zero.
Figure 1.
Summable and harmonic probabilities: partial sums and the probability of a later event
. The left panel shows finite partial sums. On the right, the blue curve is the infinite tail of the squared-reciprocal p-series, bounding the probability of a later event by the union bound; the gray curve is its upper bound. For harmonic probabilities, mutually independent events have a later occurrence with probability one, while the shared-uniform counterexample to the second Borel-Cantelli lemma has tail probability . The infinite claims follow from the calculations, not from the finite plotted range. Details.
Tail event Created 2026-09-24 Updated 2026-10-14
A tail event is an event , where is the tail sigma-algebra of the chosen sequence of random variables. It concerns information available arbitrarily far along the sequence. In the measurable coordinate models described in finite-prefix invariance of tail events, any finite edit preserves whether it occurs.
This is stronger than merely needing a long waiting time. On , with denoting heads, let , taking if there is no head. The event
is not a tail event: changing only from zero to one changes membership. Under fair independent Bernoulli trials its probability is . Even the event , of probability one in this model, is not a tail event as a literal set: the all-zero sequence and the sequence with just one initial head differ in one coordinate. A probability of zero or one does not characterize tail events.
Tail measurability of limits of sample averages Created 2026-10-05 Updated 2026-10-14
Let be a sequence of finite real random variables, and write for its sample mean. For fixed and , put . Then, at each outcome,
Consequently . The right-hand limit superior is a measurable function of for every , so the extended-real random variable is measurable with respect to the tail sigma-algebra. The same holds for the limit inferior. Thus the event of convergence to a finite limit of a sequence, and the event for any specified real , are tail events.
For a worked deterministic example, let alternate , and replace its first three entries by . For every the two sample means differ by exactly , so both converge to . Finite edits can change small-sample answers while preserving the infinite limit of a sequence.
Under independence, Kolmogorov's zero–one law makes each specified convergence event have probability zero or one. It does not select which value occurs. If a finite random variable measurable with respect to the tail sigma-algebra exists, the events for rational each have probability zero or one; continuity of probability and their nesting force to be constant almost surely. For fair independent Bernoulli trials, the strong law of large numbers establishes almost surely; the zero–one conclusion alone proves neither convergence nor the value .
Tail probability Created 2026-10-03 Updated 2026-10-14
For a real random variable , an upper-tail probability has the form and a lower-tail probability has the form . Each is the probability of an event lying beyond a chosen threshold in one direction. A tail probability need not be small: the threshold determines how much of the probability distribution it includes. A concentration inequality bounds such probabilities, often for deviations from an expected value.
Endpoint conventions matter. Write for the cumulative distribution function and . The complement rule of probability gives four distinct formulas:
The strict upper-tail convention is the survival function . Its value differs from the inclusive upper tail probability by exactly the probability mass at the threshold:
A singleton with positive probability mass is an atom of a measure. Thus excludes an atom of a measure at , whereas includes it. These endpoint identities follow directly from the cumulative distribution function properties in Kyle Siegrist, Distribution and Quantile Functions: endpoint probabilities and right tails.
As the threshold increases, both upper tail probabilities are nonincreasing functions. The strict survival function is a right-continuous function; the inclusive upper tail probability takes the upper endpoint value at a downward jump, equal to the left limit of the survival function. Both tend to zero as for a real random variable, but the speed of that decay differs across probability distributions. The cumulative distribution function properties and their proof from continuity of probability measures are given in MIT 6.436J, Lecture 4: random variables and cumulative distribution function properties.
For a fair six-sided die , the discrete uniform distribution assigns probability mass to each of . At threshold four,
Between four and five, both upper tail probabilities equal . At the largest possible outcome the distinction is especially visible: , but . The diagram shows exact probabilities for this mathematical model; its dots specify which endpoints belong to each step function.
Figure 1.
Exact fair-die probabilities: including or excluding a threshold changes the tail by one atom
. Details.
If a probability distribution has a probability density function , then every singleton has zero probability mass and
For an exponential distribution of rate two, for and zero otherwise, so for and for . Hence . This tail probability is an integral of the probability density function, not its height . A mixed probability distribution retains endpoint jumps even with a smooth component: give zero probability mass and choose this exponential distribution with probability . Then , , and for .
A two-sided tail probability can measure the absolute value of displacement from a specified center . For , the two extreme events are disjoint events, so
For strict displacement , replace the last expression by . The condition prevents overlap; at adding inclusive tails double-counts any probability mass at . For a uniform distribution on , take and : the two tail probabilities are each and their sum is . Equal tails follow from this symmetric probability distribution, not from the phrase two-sided alone; for a uniform distribution on with , the two tail probabilities are zero and .
The tail-sum formula for expectation turns a whole curve of tail probabilities into an expected value. For any nonnegative random variable , without assuming a probability density function or a finite expected value,
For a proof outline, write pointwise and exchange the expected value and Lebesgue integral using Tonelli theorem; is an indicator function. The inclusive version has the same Lebesgue integral: positive probability masses occur at only a countable set of thresholds, which has Lebesgue measure zero. For a nonnegative integer-valued random variable , the tail-sum formula instead gives . The integer hypothesis is essential for these sums. The integral identity, its indicator function proof, and the integer version appear in Kyle Siegrist, Additional Properties of Expected Value: integration and summation of right tails.
For the fair die, this tail-sum formula yields , equal to the direct weighted arithmetic mean . For the rate-two exponential distribution, the area under the survival function is , its expected value. The first panel below shades an integral of the probability density function for a single threshold; the second shades an integral of the survival function across all nonnegative thresholds. These are analytical illustrations, with the displayed curves truncated at three; their exact total areas use an infinite integration range.
Figure 2.
Exponential model: one tail probability is density area, while expectation is total survival area
. Details.
Nonnegativity is essential to interpreting the upper tail probability area as : for a general real random variable that area equals the expected value of its positive part. An integrable random variable satisfies , subtracting the expected value of its negative part; infinite terms cannot be canceled. Even when is finite almost surely, its expected value can be infinite. For the shape-one Pareto distribution on , on and for , so the area is . The finite versus infinite expected value distinction is illustrated in Kyle Siegrist, Definitions and Basic Properties: exponential and Pareto expected values.
Why are tail probabilities useful? They answer threshold questions directly, and a concentration inequality can bound them without knowing the full probability distribution. For nonnegative with finite expected value, Markov inequality gives for : a large threshold relative to the expected value forces rarity. The quantile function reverses the question by choosing a threshold for a prescribed cumulative level. If with , then ; an atom of a measure may prevent equality. For the die, , with strict upper tail probability zero and inclusive upper tail probability .
A tail probability describes rarity under a specified probability distribution; it is not automatically a conditional probability after extra information. For with ,
because the threshold events are nested. Thus the die has but . Conditioning changes the sample space and renormalizes its probability measure; the same extreme outcomes can become common in that restricted population.
Tail sigma-algebra Created 2026-09-24 Updated 2026-10-14
For a sequence of random variables on a probability space , define
The decreasing sigma-algebras are generated by the listed random variables. Their intersection consists of events expressible after discarding every finite prefix. No independence is required.[ref]
The point is to distinguish a long observation record from a property of an infinite sequence. The event of seeing infinitely many heads and the event that sample means converge survive any finite change, while an exact first-head time does not. Kolmogorov's zero–one law adds the striking conclusion that under independence each tail event has probability zero or one; the tail sigma-algebra itself can still contain many nonempty sets.
Tail-sum formula Created 2026-09-24 Updated 2026-10-18
The tail-sum formula expresses an expected value as the area under a survival function. For every nonnegative random variable ,
This Lebesgue integral and the expected value may both be ; a probability density function is unnecessary. For a random variable taking values in the nonnegative integers, the corresponding sum is
The events and coincide because takes integer values. Thus the strict sum starts at zero, while the inclusive sum starts at one.[ref]
For an original example, let the probability mass function of be
Its expected value computed directly is . The strict tail probabilities at thresholds are : their sum is again . There is no probability mass at two, yet the event still has probability . The survival function has heights on the successive intervals ; each interval has length one, so its integral is the same sum.
Figure 1.
The same expected value from probability masses and survival columns
. The exact constructed probability mass function assigns to . The contribution appears in one survival column, while the contribution appears in three. Filled endpoints give the strict survival function at its jumps; open endpoints show the preceding height. The shaded area is . Details.
The indexing matters numerically. Starting the inclusive sum at zero adds , giving for this example. Starting the strict sum at one omits , giving . For a general nonnegative random variable, replacing with inside the integral is harmless: the difference is , nonzero at only a countable set of thresholds. For each positive integer , at most thresholds can have probability mass at least ; their countable union contains all thresholds with positive probability mass and has Lebesgue measure zero. This reasoning concerns an integral, not equality of the two survival function conventions at every point.
For a continuous original example, take the uniform distribution on the interval . Its survival function is for and zero for . The area is a triangle of base four and height one:
Directly integrating against the probability density function gives the same expected value, . The integral of the probability density function alone is one; it is the survival function whose area is the expected value.
Figure 2.
A uniform density has total probability one and survival area two
. Exact curves for the uniform distribution on . The probability density function has integral one, while the survival function has integral two. Splitting the survival area at gives before the cap and beyond it, the expected values of and the positive part , respectively. Details.
The proof counts layers of each outcome. For a realized value , the indicator function equals one over an interval of length , so
Tonelli theorem permits the interchange because the indicator function is nonnegative; no finite expected value is assumed. Similarly, an outcome contributes one indicator random variable for each of the events that occur:
Again Tonelli theorem applies to the nonnegative sum, or equivalently the monotone convergence theorem applies to its partial sums. This exchanges counting layers within each outcome with averaging across outcomes. The existing integer-valued hypothesis in the tail-sum formula explains what changes when the values are not integers; the tail integral formula for moments weights the layers to obtain higher moments.
For an integrable random variable that can have either sign, apply the nonnegative formula separately to its positive part and negative part :
Integrability makes both integrals finite. More generally the difference defines an extended expected value when at least one is finite; if both are infinite, subtraction does not define an expected value.[ref]
Tail probabilities are often easier to measure or bound than a whole probability density function. For a device's survival time, the survival function records the probability of still operating at each threshold; its integral gives the mean operating time. A finite observation window naturally gives the expected value of a capped nonnegative random variable. For a mathematical bound, for all implies . The tail-sum formula turns control of a whole family of events into control of an average.
Talagrand concentration inequality for certifiable functions Created 2026-09-24 Updated 2026-11-05
For a nonnegative integer-valued certifiable function of independent random variables, suppose changing one coordinate changes by at most and a value has a certificate involving at most coordinates, with . Then for ,
For a high-value input , every low-value input must disagree with its certificate on at least coordinates: replacing its certificate positions by those of would change the low value by less than but force a value at least . Uniform weights on certificate positions therefore give Talagrand convex distance at least from the low-value event. Apply Talagrand's convex distance inequality to that event.
The bound is useful when one of the two events has known nonnegligible probability, for example at a median. A certificate alone supplies no coordinate-change constant ; both hypotheses are used in the proof. The two-threshold concentration for certifiable functions article develops the resulting mean-median comparison.
Telegraph process Created 2026-09-28 Updated 2026-11-05
The Telegraph process moves at one of two opposite constant velocities and reverses direction at Poisson event times. Its position has finite propagation speed and persistent increments.