Threshold stopping of an independent Gaussian series Created 2026-10-14 Updated 2026-11-05
Let , and stop on first reaching a fixed positive threshold. If the coefficient squares are summable, uniform integrability gives the stopped mean zero, with the terminal limit used on . If the coefficient squares diverge, for each fixed . Thus . This event is unaffected by changing finitely many of the independent variables, so the Kolmogorov zero-one law makes its probability one. Hence and is at least the positive threshold; its expectation, finite or infinite, cannot be zero.
Tie probability for independent binomial counts Created 2026-10-20 Updated 2026-11-05
For independent random variables with the same binomial distribution , partition the tie event into the mutually exclusive events . independence makes each probability the square of its binomial probability, proving the formula. At , Vandermonde's identity gives . Symmetry under exchange of the two variables then gives .
Total variation distance Created 2026-09-24 Updated 2026-10-13
Total variation distance asks how much two probability distributions can disagree about the same event. For probability measures on one measurable space , its definition isThe supremum ranges over all measurable sets, so guarantees that every yes-or-no prediction has probability error at most . This metric takes values between zero and one; it is zero exactly when the two probability measures agree. This makes it useful for comparing a sampling algorithm's output with its target probability distribution, and for defining the mixing time of a Markov chain.
On a finite set, add the excess probability mass wherever exceeds . That excess is also half the sum of all absolute differences:The last expression reveals the common probability mass that the two probability distributions can share.[ref]
Markov Chains and Mixing Times, second edition by David A. Levin and others. §4.1, Proposition 4.2 and Remark 4.3, printed p. 48; §4.2, equation (4.13), printed p. 51
The coupling characterization of total variation distance turns this overlap into an actual joint construction; minimum sum of errors in a simple hypothesis test gives it a statistical interpretation. One quantity connects probability bookkeeping, comparison of sampling methods, and the limits of a statistical test.Suppose one observation comes from either or , each with prior probability . A decision rule that chooses on an event and otherwise has success probabilityOn a finite set, choosing therefore gives . This is the equal-prior version of minimum sum of errors in a simple hypothesis test; allowing a statistical test to use an auxiliary random variable cannot improve it, because for each outcome the better of the two decisions already maximizes its contribution.
For the four-outcome example, choose after and after or , with either decision allowed after . Choosing after yields . The best success probability is 65%, with a 35% error probability. A smaller total variation distance means the two sources are harder to distinguish from one observation. Unequal prior probabilities, unequal costs, or multiple observations change this particular decision problem; the figure is not a universal bound for every experiment.
Total variation distance on a finite set 2026-10-14
Consider the probability mass functions and on the finite set . Their difference is . ThusThe event attains this difference: and . Combining and gives the same absolute discrepancy with the opposite sign. Adding to either event changes nothing, because its probability mass agrees under both probability distributions.
The factor is necessary because : every unit of excess probability mass must match a unit of deficit. For any event, including a negative term cannot increase its signed discrepancy, while omitting a positive term cannot increase it either. Therefore attains the positive maximum. This is a short way to compute an apparent optimization over subsets using one pass through .
For probability density functions relative to the same measure , the corresponding identity is . Such densities can always be taken as Radon-Nikodym derivatives relative to . Beware the notation: the total variation norm of a measure applied to the signed measure equals . A formula using that norm without its factor is using a different normalization.
Applying the same measurable function to observations from two probability distributions cannot increase their total variation distance. Every output event is tested through its preimage , which was already one of the input events. ThereforeFor the four-outcome example, recording only whether the result is makes the output probability distributions identical: each is a Bernoulli distribution with parameter . The original total variation distance was , but this coarse measurement leaves zero discrepancy. Discarding information can make two sources less distinguishable.
The same contraction holds for a common Markov kernel : . On a finite set, expand , apply the triangle inequality to each output component, and use that each row of sums to one. A common randomized postprocessing step cannot increase distinguishability either.
Transition-completion time in a Markov chain Created 2026-10-07 Updated 2026-11-05
For a Markov chain with transition matrix , fix a directed transition with . Its transition-completion time isIt counts the successful final step. Merely visiting by another route does not complete this event. In a finite irreducible chain its mean is finite and first-step analysis givesThe subtracted continuation term removes exactly the case where the designated transition has just succeeded. An equivalent construction sends that transition to a new absorbing success state and computes its Expected hitting time.
For the two-state chain , , with , the waiting time for has means and . Starting at one, the hitting time of state one is zero, whereas the transition-completion time must first wait for and then . This example makes the difference between an edge event and a state event explicit.
Truncated distribution Created 2026-09-24 Updated 2026-11-05
A truncated distribution is the conditional probability distribution obtained by restricting a random variable to a specified event with positive probability. If its original law is , the truncated law isThe denominator renormalizes the retained probability mass to one. If has a probability density function , the truncated density is ; the same restriction and normalization apply to a probability mass function.
For example, truncate a uniform distribution on to values at least six. The retained event has probability , so the new density is on , and its mean is eight. This is the law obtained by repeatedly sampling from the original law and rejecting values outside the retained set; rejection sampling becomes inefficient when the retained event is rare.
A zero-probability restriction cannot be normalized by the displayed formula. Conditioning a continuous random variable to equal a specified value requires an additional conditional-law construction or limiting convention, rather than division by zero.
Two-toss sample space 2026-10-14
Model two fair independent coin tosses, recording their order. Here denotes heads and tails, andThe power set includes every question expressible as a subset of these four outcomes. The mass comes from the fairness and independence assumptions. These are exact model probabilities, rather than measured toss frequencies.
Let be the event that the first toss is heads and the event that the second toss is heads. ThenThe union means “at least one head,” while the intersection means “both heads.” Their probabilities are and . Counting the union through the inclusion-exclusion principle gives ; alternatively its complement of an event is , of probability .
Four ordered outcomes and the event of at least one head
. Rows record the first toss and columns the second. Each cell has probability under fair independent tosses. The highlighted union includes , , and ; the overlap contains only . Details.On the same sample space, assigning masses to would still make each toss fair: . But , so the tosses would be dependent. The event of at least one head would have probability . Changing the probability measure changes the answer even when the sample space and the single-toss probabilities stay the same.
Uniform convergence on compacts in probability under an absolutely continuous measure change Created 2026-10-05 Updated 2026-11-05
If , uniform convergence on compacts in probability under implies the same convergence under . Indeed, let be the Radon-Nikodym derivative. For every event ,Integrability makes the second term arbitrarily small for large . Thus implies ; apply this to each event where the supremum on exceeds a fixed tolerance. Only absolute continuity of measures is needed, not equivalence.
Uniform integrability of conditional expectations Created 2026-10-05 Updated 2026-11-05
For one integrable random variable , the family , over arbitrary sigma-algebras , has uniform integrability. On , the defining property of conditional expectation gives , while by the Markov inequality. The integrable random variable has uniformly small absolute integrals over events of sufficiently small Probability.
Usual conditions for a filtration Created 2026-10-05 Updated 2026-11-05
The usual conditions are completeness and right continuity of a filtration: contains all subsets of null events in the ambient probability space, and . They allow the standard continuous-time martingale, stopping time and stochastic integration theorems to be used without repeated augmentation qualifications. An absolute continuity of measures change preserves old null sets but can introduce new ones; the usual completion under the new measure may therefore be understood when needed.
Waiting-time distribution Created 2026-10-25 Updated 2026-11-05
A waiting-time distribution is the probability distribution of a nonnegative random duration until a specified event. It can be discrete, absolutely continuous, or mixed. Its survival function is and its cumulative distribution function is .
If the finite-time law is absolutely continuous, the probability density function is almost everywhere. Mere continuity of the cumulative distribution function is insufficient for this density representation. At times with positive survival, the hazard function is ; it measures the instantaneous event rate conditional on having waited that long.
Constant hazard with gives , the exponential distribution. Waiting a discrete number of independent Bernoulli trials instead gives a geometric distribution. A fixed deterministic wait has a point mass at its prescribed duration rather than an ordinary density.
Sequential independent waiting stages add, so their probability distributions combine by convolution. The two-step catalytic waiting time gives a concrete chemical-kinetics example. Waiting-time distributions also describe first-passage times and time-to-event observations in survival analysis.
Wald's equation Created 2026-10-03 Updated 2026-10-14
Wald's equation turns a random sum into a product of two expected values: the average contribution per step and the average number of steps. Its surprising feature is that the number of steps may depend on what has already happened. A procedure can stop on its first successful Bernoulli trial, for example, while the identity still accounts for its total cost. The crucial restriction is that the decision to take the next step cannot use that step's unobserved outcome.
Let be real independent and identically distributed random variables with , let , and put and . Suppose is a nonnegative integer-valued stopping time for the natural filtration , with . Then is an integrable random variable andThese hypotheses allow statistical dependence between and the increments; they do not allow access to future increments. The finite-expected value requirement is stronger than stopping almost surely.[ref]
For the proof idea, count step only on . This event is determined by the first observations, so its indicator function is independent of . The Proof of Wald's equation by stopping indicators combines this observation with the tail-sum formula for expectation. The same calculation with gives , which justifies interchanging the expected value and the infinite series. This is why the result survives a data-dependent end point without assuming an independent count.
The identity is useful in randomized algorithms and repeated sampling: one can translate an average amount of work into an average number of attempts, or conversely, without first finding the full probability distribution of the total. The examples below separate a permissible stopping rule from a rule that peeks ahead and from a valid rule whose average waiting time is infinite.
Zero-event binomial upper confidence bound Created 2026-10-14 Updated 2026-11-05
If and is observed, the maximum-likelihood estimate of is zero. A one-sided upper confidence bound solves , so . For large , , giving approximately at 95% confidence. This describes uncertainty after no observed events, rather than proving that the underlying probability is zero. It assumes independent observations with a common event probability; clustered data require a different uncertainty calculation.
Zero-truncated claim-count distribution Created 2026-10-06 Updated 2026-11-05
For a nonnegative integer claim count with , the conditional law given has masses for . Its probability generating function is . Truncation at zero is meaningful only when the positive-count event has positive probability.

