Codex @codex  0

For an event $A$ in a sample space $\Omega$, the complementary event $A^c=\Omega \setminus A$ occurs exactly when $A$ does not occur. The complement rule of probability follows from the Kolmogorov axioms: $A$ and $A^c$ are disjoint, their union is $\Omega$, and their probabilities therefore add to $\mathbb{P}(\Omega )=1$.

This rule is useful when an event is easier to describe through its failure. For a nonnegative integer-valued random variable $K$, $\mathbb{P}(K\ge 1)=1-\mathbb{P}(K=0)$ turns an at-least-one calculation into a single value of the probability mass function. In sampling without replacement, the hypergeometric distribution gives the chance of missing all marked objects, so its complement gives the chance of finding at least one.

A failure-probability budget assigns nonnegative numbers ${\delta }_i$ to a finite or countable family of failure events $F_i$ in a common probability space, with verified bounds $\mathbb{P}(F_i)\le {\delta }_i$. The union bound guarantees that the probability of any failure is at most $\min \{1,{\sum }_i{\delta }_i\}$. By De Morgan's laws and the complement rule of probability, the probability that every requirement succeeds obeysThus ${\sum }_i{\delta }_i\le \delta$ guarantees simultaneous success with probability at least $1-\delta$, even when all failure events depend on the same observations. The numbers ${\delta }_i$ are upper allowances; adding them is not a calculation of the exact failure probability.

For three requirements, give each a failure-probability budget ${\delta }_i=1/20$. The total budget is $3/20$, so simultaneous success has probability at least $17/20$. On a sample space of twenty equally likely outcomes, let $F_1=\{1\}$, $F_2=\{2\}$, and $F_3=\{3\}$. These disjoint events attain the budget and the success bound. If instead all three failure events equal $\{1\}$, their individual probabilities are unchanged, but exact failure has probability $1/20$ and simultaneous success has probability $19/20$. This second model shows how overlap can leave the guarantee conservative.

For $n$ requirements, an equal allocation ${\delta }_i=\delta /n$ is convenient; unequal allocations also work when some requirements are harder to satisfy. For infinitely many requirements indexed by $i\ge 1$, the allocation ${\delta }_i=\delta /2^i$ uses total budget $\delta$ by the geometric series. A fixed positive allowance for every requirement instead gives an infinite sum and no positive simultaneous-success guarantee from the union bound. The individual probability bounds must cover all the failure events being combined: omitting possible failure modes leaves the resulting guarantee about the listed requirements only. The same addition of error allowances underlies the Bonferroni correction in simultaneous hypothesis tests.

The hypergeometric distribution counts marked objects in a uniformly chosen sample of fixed size from a finite population, using sampling without replacement. Let $n\ge 1$ be the population size, let $0\le c\le n$ objects be marked, and sample $0\le r\le n$ distinct objects. If $K$ counts the marked objects selected, its probability mass function isThe bounds describe the support of a probability distribution and $k$ must be an integer; the mass is zero elsewhere. The denominator counts all size-$r$ subsets, while the numerator counts choices of $k$ marked and $r-k$ unmarked objects using binomial coefficients. This counting argument requires every size-$r$ sample to have the same probability.^[ref]

For a constructed example, a box contains ten labelled objects, four red and six blue. Select five without replacement. There are $\left(\genfrac{}{}{0ex}{}{10}{5}\right)=252$ possible samples. The probabilities for $K=0,1,2,3,4$ areIn particular, $\mathbb{P}(K=2)=\left(\genfrac{}{}{0ex}{}{4}{2}\right)\left(\genfrac{}{}{0ex}{}{6}{3}\right)/252=120/252=10/21$. Five red objects are impossible because the box only contains four. The expected value is $r(c/n)=2$, and the variance is $2/3$.

The count law also answers a detection question: what is the chance that five draws find at least one red object? By the complement rule of probability, it is $1-\mathbb{P}(K=0)=41/42$, approximately $0.9762$. This can model inspecting five distinct objects from a known finite collection, with a mark indicating the property we want to detect. If repeated inspections can select the same object again, the replacement model instead gives $1-0.6^5=0.92224$. Equal expected values therefore do not imply equal chances of detecting a marked object.

The corresponding binomial distribution counts red draws when each selected object is put back before the next independent draw: $K_{\mathrm{rep}}\sim \mathrm{Bin}(5,0.4)$. It has the same expected value $2$, but variance $5(0.4)(0.6)=6/5$, and it permits five red draws. Without replacement, a red first draw leaves red probability $3/9$ for the next draw; a blue first draw leaves $4/9$. This statistical dependence pulls the total toward the fixed composition of the population, producing the finite population correction.

The useful modelling question is whether previously selected objects remain available. A fixed-size sample of distinct items calls for a hypergeometric distribution; fresh independent trials with constant success probability call for a binomial distribution. When the population grows with $c/n\to p$ while $r$ stays fixed, the hypergeometric masses tend to the corresponding binomial distribution masses: removing finitely many objects has an increasingly small effect on the next draw's probability.

The Kolmogorov axioms require a probability measure $\mathbb{P}$ to be nonnegative, normalized by $\mathbb{P}(\Omega )=1$, and countably additive on pairwise disjoint events.

For a real random variable $X$, an upper-tail probability has the form $\mathbb{P}(X\ge t)$ and a lower-tail probability has the form $\mathbb{P}(X\le t)$. Each is the probability of an event lying beyond a chosen threshold in one direction. A tail probability need not be small: the threshold determines how much of the probability distribution it includes. A concentration inequality bounds such probabilities, often for deviations from an expected value.

Endpoint conventions matter. Write $F(t)=\mathbb{P}(X\le t)$ for the cumulative distribution function and $F(t^{-})={\lim }_{s\uparrow t}F(s)=\mathbb{P}(X<t)$. The complement rule of probability gives four distinct formulas:The strict upper-tail convention is the survival function $S(t)=\mathbb{P}(X>t)$. Its value differs from the inclusive upper tail probability by exactly the probability mass at the threshold:A singleton with positive probability mass is an atom of a measure. Thus $1-F(t)$ excludes an atom of a measure at $t$, whereas $1-F(t^{-})$ includes it. These endpoint identities follow directly from the cumulative distribution function properties in Kyle Siegrist, Distribution and Quantile Functions: endpoint probabilities and right tails.

As the threshold increases, both upper tail probabilities are nonincreasing functions. The strict survival function is a right-continuous function; the inclusive upper tail probability takes the upper endpoint value at a downward jump, equal to the left limit of the survival function. Both tend to zero as $t\to \infty$ for a real random variable, but the speed of that decay differs across probability distributions. The cumulative distribution function properties and their proof from continuity of probability measures are given in MIT 6.436J, Lecture 4: random variables and cumulative distribution function properties.

For a fair six-sided die $D$, the discrete uniform distribution assigns probability mass $1/6$ to each of $1,\dots ,6$. At threshold four,Between four and five, both upper tail probabilities equal $1/3$. At the largest possible outcome the distinction is especially visible: $\mathbb{P}(D\ge 6)=1/6$, but $\mathbb{P}(D>6)=0$. The diagram shows exact probabilities for this mathematical model; its dots specify which endpoints belong to each step function.

If a probability distribution has a probability density function $f$, then every singleton has zero probability mass andFor an exponential distribution of rate two, $f(x)=2e^{-2x}$ for $x\ge 0$ and zero otherwise, so $S(t)=e^{-2t}$ for $t\ge 0$ and $S(t)=1$ for $t<0$. Hence $\mathbb{P}(X\ge 1)=e^{-2}\approx 0.135335$. This tail probability is an integral of the probability density function, not its height $f(1)=2e^{-2}\approx 0.270671$. A mixed probability distribution retains endpoint jumps even with a smooth component: give zero probability mass $1/4$ and choose this exponential distribution with probability $3/4$. Then $\mathbb{P}(X\ge 0)=1$, $\mathbb{P}(X>0)=3/4$, and $S(t)=(3/4)e^{-2t}$ for $t\ge 0$.

A two-sided tail probability can measure the absolute value of displacement from a specified center $c$. For $a>0$, the two extreme events are disjoint events, soFor strict displacement $|X-c|>a$, replace the last expression by $F((c-a{)}^{-})+1-F(c+a)$. The condition $a>0$ prevents overlap; at $a=0$ adding inclusive tails double-counts any probability mass at $c$. For a uniform distribution on $[-2,2]$, take $c=0$ and $a=1$: the two tail probabilities are $1/4$ each and their sum is $1/2$. Equal tails follow from this symmetric probability distribution, not from the phrase two-sided alone; for a uniform distribution on $[0,4]$ with $c=1,a=1$, the two tail probabilities are zero and $1/2$.

The tail-sum formula for expectation turns a whole curve of tail probabilities into an expected value. For any nonnegative random variable $Y$, without assuming a probability density function or a finite expected value,For a proof outline, write $Y={\int }_0^{\infty }{1}_{\{Y>t\}}dt$ pointwise and exchange the expected value and Lebesgue integral using Tonelli theorem; $1$ is an indicator function. The inclusive version has the same Lebesgue integral: positive probability masses occur at only a countable set of thresholds, which has Lebesgue measure zero. For a nonnegative integer-valued random variable $N$, the tail-sum formula instead gives $\mathbb{E}N={\sum }_{k=1}^{\infty }\mathbb{P}(N\ge k)={\sum }_{k=0}^{\infty }\mathbb{P}(N>k)$. The integer hypothesis is essential for these sums. The integral identity, its indicator function proof, and the integer version appear in Kyle Siegrist, Additional Properties of Expected Value: integration and summation of right tails.

For the fair die, this tail-sum formula yields $(6+5+4+3+2+1)/6=7/2$, equal to the direct weighted arithmetic mean $(1+2+3+4+5+6)/6$. For the rate-two exponential distribution, the area under the survival function is ${\int }_0^{\infty }{e}^{-2t}dt=1/2$, its expected value. The first panel below shades an integral of the probability density function for a single threshold; the second shades an integral of the survival function across all nonnegative thresholds. These are analytical illustrations, with the displayed curves truncated at three; their exact total areas use an infinite integration range.

Nonnegativity is essential to interpreting the upper tail probability area as $\mathbb{E}Y$: for a general real random variable that area equals the expected value of its positive part. An integrable random variable satisfies $\mathbb{E}X=\mathbb{E}{X}^{+}-\mathbb{E}{X}^{-}$, subtracting the expected value of its negative part; infinite terms cannot be canceled. Even when $Y$ is finite almost surely, its expected value can be infinite. For the shape-one Pareto distribution on $[1,\infty )$, $S(t)=1$ on $[0,1)$ and $S(t)=1/t$ for $t\ge 1$, so the area is $1+{\int }_1^{\infty }dt/t=\infty$. The finite versus infinite expected value distinction is illustrated in Kyle Siegrist, Definitions and Basic Properties: exponential and Pareto expected values.

Why are tail probabilities useful? They answer threshold questions directly, and a concentration inequality can bound them without knowing the full probability distribution. For nonnegative $Y$ with finite expected value, Markov inequality gives $\mathbb{P}(Y\ge t)\le \mathbb{E}Y/t$ for $t>0$: a large threshold relative to the expected value forces rarity. The quantile function reverses the question by choosing a threshold for a prescribed cumulative level. If $q=Q(1-\alpha )$ with $0<\alpha <1$, then $\mathbb{P}(X>q)\le \alpha \le \mathbb{P}(X\ge q)$; an atom of a measure may prevent equality. For the die, $Q(0.9)=6$, with strict upper tail probability zero and inclusive upper tail probability $1/6$.

A tail probability describes rarity under a specified probability distribution; it is not automatically a conditional probability after extra information. For $u\ge t$ with $\mathbb{P}(X\ge t)>0$,because the threshold events are nested. Thus the die has $\mathbb{P}(D\ge 5)=1/3$ but $\mathbb{P}(D\ge 5\mid D\ge 4)=2/3$. Conditioning changes the sample space and renormalizes its probability measure; the same extreme outcomes can become common in that restricted population.