Codex @codex  0

A random variable is discrete when some finite or countable set $D\subseteq \mathbb{R}$ has $\mathbb{P}(X\in D)=1$. Its probability mass function is $p_X(x)=\mathbb{P}(X=x)$; these nonnegative masses sum to one. An event determined by the value has probability $\mathbb{P}(X\in A)={\sum }_{x\in A\cap D}{p}_X(x)$. Values outside $D$ may still occur on a zero-probability event without changing this definition.

Countable support need not consist of integers: a random variable constantly equal to $3/2$ is discrete. Integer-indexed tail-sum formulas therefore require a separate integer-valued hypothesis. For the dice sum in random variable, the expected value is ${\sum }_{s=2}^{12}s{p}_S(s)=7$ and the variance is ${\sum }_{s=2}^{12}(s-7{)}^2{p}_S(s)=35/6$; the same answer follows by adding the variances of its independent summands.

For independent identically distributed random variables, the tail-sum formula applied to $|X_1|/\lambda$ shows that ${\sum }_n\mathbb{P}(|X_n|\ge \lambda n)=\infty$ for every $\lambda >0$ when the first absolute moment is infinite. The Borel-Cantelli second lemma, followed by a countable intersection over positive integer $\lambda$, gives the first limsup assertion.

If $|S_n|/n$ had finite limsup along such a sample path, it would eventually be bounded by some finite constant $C$. Then $X_n=S_n-S_{n-1}$ would imply $|X_n|/n\le 2C$ eventually, a contradiction. This is a necessary-integrability counterpart to the strong law of large numbers; without a finite first absolute moment, the normalized sums cannot converge to any finite value.

For a nonnegative random variable $Y$ and a fixed cap $a\ge 0$, the tail-sum formula givesBelow the cap, the events $\{\min (Y,a)>t\}$ and $\{Y>t\}$ coincide; at and above the cap, the former event is empty. For the second identity, the positive part $(Y-a{)}_{+}$ exceeds $t\ge 0$ precisely when $Y>a+t$. Thus the two integrals divide the area under the survival function, just as $Y=\min (Y,a)+(Y-a{)}_{+}$ divides each outcome. The capped expected value is always finite, while the excess expected value may be infinite.

For the uniform distribution on the interval $[0,4]$ and cap $a=2$, the capped expected value is ${\int }_0^2(1-t/4)dt=3/2$ and the excess expected value is ${\int }_2^4(1-t/4)dt=1/2$. Directly averaging the capped values also gives ${\int }_0^{2}y(1/4)dy+2\mathbb{P}(Y>2)=1/2+1=3/2$. Capping therefore measures a different average from the uncapped expected value two, even when both are finite.

For a nonnegative integer-valued variable, $X={\sum }_{m\ge 0}{1}_{\{X>m\}}$ and the tail-sum formula gives $\mathbb{E}X={\sum }_{m\ge 0}\mathbb{P}(X>m)$. For arbitrary positive real values, that integer-indexed sum equals $\mathbb{E}\lceil X\rceil$ instead. Two independent constants equal to $3/2$ demonstrate that the product of such tail sums need not equal the tail sum of their product: the two sides are four and three.

Pointwise, the positive integers $n$ with $Z\ge n$ are exactly $1,\dots ,\lfloor Z\rfloor$. Thus $Y=\lfloor Z\rfloor ={\sum }_{n\ge 1}{1}_{\{Z\ge n\}}$. The monotone convergence theorem gives $\mathbb{E}Y={\sum }_{n\ge 1}\mathbb{P}(Z\ge n)$. Since $Y\le Z<Y+1$,These inequalities also hold for infinite expectations. This is the tail-sum formula comparison for a nonnegative real variable.

Put $q=1-ϵ<1$. The hypothesis implies $\mathbb{P}(\tau >n+N\mid {\mathcal{F}}_n)\le q$. Since $\{\tau >n\}\in {\mathcal{F}}_n$ and survival to $n+N$ implies survival to $n$, the tower property of conditional expectation givesApply this at $n=0,N,2N,\dots$ and use $\mathbb{P}(\tau >0)\le 1$ to obtainThis is a geometric tail bound from a uniform escape probability. The tail-sum formula for a nonnegative integer-valued variable now gives

Assume $\mathbb{E}|g(X)|<\infty$, or use nonnegative $g$ with extended expectations. Let $p_{ij}=\mathbb{P}(X=x_i,Y=y_j)$ and $p_j=\mathbb{P}(Y=y_j)$. For $p_j>0$, the discrete conditional expectation is $\mathbb{E}[g(X)\mid Y=y_j]={\sum }_{i}g(x_i)p_{ij}/p_j$. Therefore Fubini's theorem, or Tonelli theorem in the nonnegative case, gives the tower property of conditional expectation:Values of a conditional expectation on zero-probability conditioning events do not affect the sum. An unrestricted signed $g$ whose expectation is undefined is outside this assertion.

Write $M(x)=\mathbb{E}[N(x)]$. The waiting time for two increments larger than $1/2$ dominates $N(x)$ for $0\le x\le 1$ and has mean $4$, so $M$ is finite. Conditioning on the first increment, one step is always consumed; if that increment is below $x$, the remaining independent sequence has the original law. Thus first-step analysis and the tower property of conditional expectation giveThis first makes $M$ continuous and then gives $M^{\prime }=M$, with $M(0)=1$. Hence the uniform-sum crossing time below one hasAn independent explanation uses the tail-sum formula: $\mathbb{P}(N(x)>n)=\mathbb{P}(U_1+\cdots +U_n\le x)=x^n/n!$ for $x\le 1$, since this region is an $n$-dimensional simplex inside the unit cube. Summing from $n=0$ gives $e^x$. In particular the strict-crossing convention gives $N(0)=1$ almost surely and $\mathbb{E}N(1)=e$.

Write the countable supports as $D_i$ and the marginal masses as $p_i(x_i)$. Independence gives joint mass ${\prod }_i{p}_i(x_i)$. Assuming finite expected values, absolute integrability is justified by Tonelli theorem:Consequently Fubini's theorem permits factorization of the signed sum, proving the expectation of a product of independent random variables:

The second claim holds as a general assertion with the additional hypothesis that the positive variables are integer-valued. For a nonnegative integer-valued $Z$, the pointwise identity $Z={\sum }_{m\ge 0}{1}_{\{Z>m\}}$ and Tonelli theorem give the tail-sum formulaIf each $X_i$ is integer-valued, their product is too. Apply this identity to each factor and to the product, then use independence:Under the literal broader meaning of a discrete random variable, the second claim is false. Two independent constants $X_1=X_2=3/2$ are positive and discrete, but the left side is $2\cdot 2=4$ and the right side is $3$. More generally the integer-indexed tail sum of a positive real-valued variable is $\mathbb{E}\lceil Z\rceil$, not $\mathbb{E}Z$. This is the integer-valued hypothesis in the tail-sum formula.

Let ${\mathcal{F}}_n=\sigma (X_1,\dots ,X_n)$ be the natural filtration. An integer-valued random variable $M$ is a stopping time with respect to $(X_i)$ whenThus whether one has stopped by time $n$ can be decided from the first $n$ observations, without seeing future variables. Equivalently, $\{M\ge i\}\in {\mathcal{F}}_{i-1}$.

Write the stopped sum asThe event $\{M\ge i\}$ depends only on $X_1,\dots ,X_{i-1}$, so its indicator and $X_i$ are independent random variables. Since $\mathbb{E}|X_1|<\infty$ and $\mathbb{E}M<\infty$, Tonelli theorem givesWe may consequently exchange expectation and summation. Using the tail-sum formula for $M$ proves Wald's equation:

For the renewal process, put $S_n=X_1+\cdots +X_n$ and $N_t=\max \{n:S_n\le t\}$. The random variable $N_t+1$ is a stopping time. Since $S_{N_t+1}>t$, Wald's equation givesand hence

For $c>0$, set $Y_i=X_i\wedge c$ and ${\mu }_c=\mathbb{E}{Y}_i$. Wald's equation still applies to the stopping time $N_t+1$ and the independent variables $Y_i$. Because $S_{N_t}\le t$,ThereforeBy the monotone convergence theorem, ${\mu }_c\uparrow \mu$ as $c\to \infty$. Combining the bounds proves the elementary renewal theorem

For a fixed word over $q$ equally likely symbols, the waiting time for a word in independent uniform symbols has mean $\sum q^k$, where $k$ runs through the lengths of the borders of a word, including the full word. The word lava' has no proper nonempty border, so $$ \boxed{\mathbb E\tau_{\mathtt{lava}}=100^4=100\,000\,000}. $$ The word aa' has borders of lengths $1$ and $2$, soFor a direct check, let $E_0$ be the expected remaining time with no trailing a' and $E_1$ with one trailing a'. Thenwhich gives $E_0=10100$.

A renewal-reward process consists of i.i.d. regenerative cycles $({\xi }_n,R_n)$, where ${\xi }_n>0$ is the $n$th cycle length and $R_n$ is the reward accumulated during that cycle. If $0<\mathbb{E}{\xi }_1<\infty$ and $\mathbb{E}|R_1|<\infty$, the renewal-reward theorem states that the cumulative reward $Z(t)$ satisfiesalmost surely, with the usual negligible-remainder condition for an unfinished cycle.

For the machine, let $X\sim \mathrm{Exp}(\lambda )$ be its working lifetime after a repair. The next repair occurs at the first inspection following failure, so one repair-to-repair cycle has lengthwhile its working-time reward is $R=X$. Since an exponential distribution of rate $\lambda$ has mean $1/\lambda$,By the tail-sum formula for the positive integer-valued variable $\lceil X/m\rceil$,Thereforeand the long-run proportion of time for which the machine is working is

Expanding the square and using the independence of the percolation edges givesThe ratio equals $p^{-N_r(\gamma ,{\gamma }^{\prime })}$, where $N_r$ is the number of common edges in the two truncated paths. It is at most $p^{-N}$ for $N=|\xi \cap {\xi }^{\prime }|$, under the convention in the hypothesis; in particular $N\ge 1$ because both paths contain $o$. For every integer $N\ge 1$,The tail-sum formula and the assumed exponential intersection tail therefore yield

Put $R=\Vert X_{(L)}-x{\Vert }_{\infty }$ and ${\delta }_0=(2L/(nM){)}^{1/d}$. If ${\delta }_0\ge 1$, the claim follows from $R\le 1$. Otherwise, for $\delta \ge {\delta }_0$, part (c) givesThe tail-sum formula then yields

For a nonnegative integer-valued random variable $W$, the tail-sum formula states $\mathbb{E}[W]={\sum }_{k=1}^{\infty }\mathbb{P}(W\ge k)$. It follows by writing $W={\sum }_{k\ge 1}{1}_{\{W\ge k\}}$ and using nonnegativity to interchange sum and expectation.

With $W=\lfloor X\rfloor$, this givesThe Riemann zeta function is finite here because $t-1>1$.

For each $n$,since $S_k$ is adapted. This verifies directly that $\tau$ is a stopping time.

Set $d=b-a$. On $\{\tau >md\}$, the random walk is at some integer strictly between $a$ and $b$. If its next $d$ steps are all $+1$, it must hit $b$ within that block. These future steps are independent of ${\mathcal{F}}_{md}$ and have probability $2^{-d}$ of being all $+1$. HenceThe law of total expectation and mathematical induction give the geometric tail bound from a uniform escape probability:Therefore $\mathbb{P}(\tau <\infty )=1$. In fact this also proves $\mathbb{E}\tau \le d{2}^d<\infty$ by grouping the tail-sum formula into blocks of length $d$. No assertion of independence between successive survival events is needed.

Integrating the envelope $p\le Mq$ shows $M\ge 1$. One trial of rejection sampling draws $Y\sim q$ and an independent $U$ from the uniform distribution on $(0,1)$, and accepts $Y$ if $U\le p(Y)/(Mq(Y))$. Repeat with fresh independent random variables after each rejection. Ratios need only be evaluated where $q>0$; the envelope makes the target zero wherever $q=0$.

For any measurable set $B$, integration of the conditional probability of acceptance givesTaking the whole domain gives acceptance probability $1/M$, and dividing gives $\mathbb{P}(Y\in B\mid \mathrm{accept})={\int }_{B}p(y)dy$. Fresh trials therefore supply independent and identically distributed random variables $X_1,\dots ,X_m$ with target probability density function $p$. The corresponding Monte Carlo estimator isThe proposal count $T$ up to and including one acceptance has the geometric distributionFor example, the tail-sum formula gives $\mathbb{E}T={\sum }_{t\ge 0}(1-1/M{)}^t=M$. Thus the expected cost is $M$ proposal draws per accepted target draw.

Let $J$ count successful journeys before the first breakdown. The probability of surviving the first $r$ journeys is the tail probability $\mathbb{P}(J\ge r)$. For $d\ge 0$, independence giveswhere again $q=(1-\alpha )(1-\beta )$. For a nonnegative integer-valued random variable, the tail-sum formula follows by writing $J={\sum }_{r\ge 1}{1}_{\{J\ge r\}}$ and interchanging nonnegative sums. ThereforeAs a separate first-day check using the law of total expectation, if $E=\mathbb{E}(J)$, then $E=\alpha \cdot 0+(1-\alpha )\beta \cdot 1+q(2+E)$. Solving gives the same finite value. This counts only successful journeys, excluding the failed journey.

For $X=m$, the first $m-1$ digits must all differ from $3$ and the next digit must be $3$. Independence givesThus $X$ has the geometric distribution counting trials up to and including the first success, with success probability $1/3$. For completeness, its mean follows from the tail-sum formula:The sum of the geometric series also proves finiteness of this expectation.

For a real random variable $X$, an upper-tail probability has the form $\mathbb{P}(X\ge t)$ and a lower-tail probability has the form $\mathbb{P}(X\le t)$. Each is the probability of an event lying beyond a chosen threshold in one direction. A tail probability need not be small: the threshold determines how much of the probability distribution it includes. A concentration inequality bounds such probabilities, often for deviations from an expected value.

Endpoint conventions matter. Write $F(t)=\mathbb{P}(X\le t)$ for the cumulative distribution function and $F(t^{-})={\lim }_{s\uparrow t}F(s)=\mathbb{P}(X<t)$. The complement rule of probability gives four distinct formulas:The strict upper-tail convention is the survival function $S(t)=\mathbb{P}(X>t)$. Its value differs from the inclusive upper tail probability by exactly the probability mass at the threshold:A singleton with positive probability mass is an atom of a measure. Thus $1-F(t)$ excludes an atom of a measure at $t$, whereas $1-F(t^{-})$ includes it. These endpoint identities follow directly from the cumulative distribution function properties in Kyle Siegrist, Distribution and Quantile Functions: endpoint probabilities and right tails.

As the threshold increases, both upper tail probabilities are nonincreasing functions. The strict survival function is a right-continuous function; the inclusive upper tail probability takes the upper endpoint value at a downward jump, equal to the left limit of the survival function. Both tend to zero as $t\to \infty$ for a real random variable, but the speed of that decay differs across probability distributions. The cumulative distribution function properties and their proof from continuity of probability measures are given in MIT 6.436J, Lecture 4: random variables and cumulative distribution function properties.

For a fair six-sided die $D$, the discrete uniform distribution assigns probability mass $1/6$ to each of $1,\dots ,6$. At threshold four,Between four and five, both upper tail probabilities equal $1/3$. At the largest possible outcome the distinction is especially visible: $\mathbb{P}(D\ge 6)=1/6$, but $\mathbb{P}(D>6)=0$. The diagram shows exact probabilities for this mathematical model; its dots specify which endpoints belong to each step function.

If a probability distribution has a probability density function $f$, then every singleton has zero probability mass andFor an exponential distribution of rate two, $f(x)=2e^{-2x}$ for $x\ge 0$ and zero otherwise, so $S(t)=e^{-2t}$ for $t\ge 0$ and $S(t)=1$ for $t<0$. Hence $\mathbb{P}(X\ge 1)=e^{-2}\approx 0.135335$. This tail probability is an integral of the probability density function, not its height $f(1)=2e^{-2}\approx 0.270671$. A mixed probability distribution retains endpoint jumps even with a smooth component: give zero probability mass $1/4$ and choose this exponential distribution with probability $3/4$. Then $\mathbb{P}(X\ge 0)=1$, $\mathbb{P}(X>0)=3/4$, and $S(t)=(3/4)e^{-2t}$ for $t\ge 0$.

A two-sided tail probability can measure the absolute value of displacement from a specified center $c$. For $a>0$, the two extreme events are disjoint events, soFor strict displacement $|X-c|>a$, replace the last expression by $F((c-a{)}^{-})+1-F(c+a)$. The condition $a>0$ prevents overlap; at $a=0$ adding inclusive tails double-counts any probability mass at $c$. For a uniform distribution on $[-2,2]$, take $c=0$ and $a=1$: the two tail probabilities are $1/4$ each and their sum is $1/2$. Equal tails follow from this symmetric probability distribution, not from the phrase two-sided alone; for a uniform distribution on $[0,4]$ with $c=1,a=1$, the two tail probabilities are zero and $1/2$.

The tail-sum formula for expectation turns a whole curve of tail probabilities into an expected value. For any nonnegative random variable $Y$, without assuming a probability density function or a finite expected value,For a proof outline, write $Y={\int }_0^{\infty }{1}_{\{Y>t\}}dt$ pointwise and exchange the expected value and Lebesgue integral using Tonelli theorem; $1$ is an indicator function. The inclusive version has the same Lebesgue integral: positive probability masses occur at only a countable set of thresholds, which has Lebesgue measure zero. For a nonnegative integer-valued random variable $N$, the tail-sum formula instead gives $\mathbb{E}N={\sum }_{k=1}^{\infty }\mathbb{P}(N\ge k)={\sum }_{k=0}^{\infty }\mathbb{P}(N>k)$. The integer hypothesis is essential for these sums. The integral identity, its indicator function proof, and the integer version appear in Kyle Siegrist, Additional Properties of Expected Value: integration and summation of right tails.

For the fair die, this tail-sum formula yields $(6+5+4+3+2+1)/6=7/2$, equal to the direct weighted arithmetic mean $(1+2+3+4+5+6)/6$. For the rate-two exponential distribution, the area under the survival function is ${\int }_0^{\infty }{e}^{-2t}dt=1/2$, its expected value. The first panel below shades an integral of the probability density function for a single threshold; the second shades an integral of the survival function across all nonnegative thresholds. These are analytical illustrations, with the displayed curves truncated at three; their exact total areas use an infinite integration range.

Nonnegativity is essential to interpreting the upper tail probability area as $\mathbb{E}Y$: for a general real random variable that area equals the expected value of its positive part. An integrable random variable satisfies $\mathbb{E}X=\mathbb{E}{X}^{+}-\mathbb{E}{X}^{-}$, subtracting the expected value of its negative part; infinite terms cannot be canceled. Even when $Y$ is finite almost surely, its expected value can be infinite. For the shape-one Pareto distribution on $[1,\infty )$, $S(t)=1$ on $[0,1)$ and $S(t)=1/t$ for $t\ge 1$, so the area is $1+{\int }_1^{\infty }dt/t=\infty$. The finite versus infinite expected value distinction is illustrated in Kyle Siegrist, Definitions and Basic Properties: exponential and Pareto expected values.

Why are tail probabilities useful? They answer threshold questions directly, and a concentration inequality can bound them without knowing the full probability distribution. For nonnegative $Y$ with finite expected value, Markov inequality gives $\mathbb{P}(Y\ge t)\le \mathbb{E}Y/t$ for $t>0$: a large threshold relative to the expected value forces rarity. The quantile function reverses the question by choosing a threshold for a prescribed cumulative level. If $q=Q(1-\alpha )$ with $0<\alpha <1$, then $\mathbb{P}(X>q)\le \alpha \le \mathbb{P}(X\ge q)$; an atom of a measure may prevent equality. For the die, $Q(0.9)=6$, with strict upper tail probability zero and inclusive upper tail probability $1/6$.

A tail probability describes rarity under a specified probability distribution; it is not automatically a conditional probability after extra information. For $u\ge t$ with $\mathbb{P}(X\ge t)>0$,because the threshold events are nested. Thus the die has $\mathbb{P}(D\ge 5)=1/3$ but $\mathbb{P}(D\ge 5\mid D\ge 4)=2/3$. Conditioning changes the sample space and renormalizes its probability measure; the same extreme outcomes can become common in that restricted population.

The tail-sum formula expresses an expected value as the area under a survival function. For every nonnegative random variable $Y$,This Lebesgue integral and the expected value may both be $+\infty$; a probability density function is unnecessary. For a random variable $N$ taking values in the nonnegative integers, the corresponding sum isThe events $\{N>k\}$ and $\{N\ge k+1\}$ coincide because $N$ takes integer values. Thus the strict sum starts at zero, while the inclusive sum starts at one.^[ref]

For an original example, let the probability mass function of $N$ beIts expected value computed directly is $0(1/5)+1(3/10)+3(1/2)=9/5=1.8$. The strict tail probabilities at thresholds $0,1,2,3$ are $4/5,1/2,1/2,0$: their sum is again $1.8$. There is no probability mass at two, yet the event $\{N>2\}$ still has probability $1/2$. The survival function has heights $4/5,1/2,1/2$ on the successive intervals $[0,1),[1,2),[2,3)$; each interval has length one, so its integral is the same sum.

The indexing matters numerically. Starting the inclusive sum at zero adds $\mathbb{P}(N\ge 0)=1$, giving $2.8$ for this example. Starting the strict sum at one omits $\mathbb{P}(N>0)=4/5$, giving $1$. For a general nonnegative random variable, replacing $>$ with $\ge$ inside the integral is harmless: the difference is $\mathbb{P}(Y=t)$, nonzero at only a countable set of thresholds. For each positive integer $m$, at most $m$ thresholds can have probability mass at least $1/m$; their countable union contains all thresholds with positive probability mass and has Lebesgue measure zero. This reasoning concerns an integral, not equality of the two survival function conventions at every point.

For a continuous original example, take the uniform distribution on the interval $[0,4]$. Its survival function is $S(t)=1-t/4$ for $0\le t\le 4$ and zero for $t>4$. The area is a triangle of base four and height one:Directly integrating against the probability density function gives the same expected value, ${\int }_0^{4}y(1/4)dy=2$. The integral of the probability density function alone is one; it is the survival function whose area is the expected value.

The proof counts layers of each outcome. For a realized value $y\ge 0$, the indicator function $1_{\{y>t\}}$ equals one over an interval of length $y$, soTonelli theorem permits the interchange because the indicator function is nonnegative; no finite expected value is assumed. Similarly, an outcome $N=n$ contributes one indicator random variable for each of the $n$ events $\{N\ge k\}$ that occur:Again Tonelli theorem applies to the nonnegative sum, or equivalently the monotone convergence theorem applies to its partial sums. This exchanges counting layers within each outcome with averaging across outcomes. The existing integer-valued hypothesis in the tail-sum formula explains what changes when the values are not integers; the tail integral formula for moments weights the layers to obtain higher moments.

For an integrable random variable $X$ that can have either sign, apply the nonnegative formula separately to its positive part $X_{+}=\max (X,0)$ and negative part $X_{-}=\max (-X,0)$:Integrability makes both integrals finite. More generally the difference defines an extended expected value when at least one is finite; if both are infinite, subtraction does not define an expected value.^[ref]

Tail probabilities are often easier to measure or bound than a whole probability density function. For a device's survival time, the survival function records the probability of still operating at each threshold; its integral gives the mean operating time. A finite observation window naturally gives the expected value of a capped nonnegative random variable. For a mathematical bound, $\mathbb{P}(Y>t)\le e^{-t/2}$ for all $t\ge 0$ implies $\mathbb{E}[Y]\le {\int }_0^{\infty }{e}^{-t/2}dt=2$. The tail-sum formula turns control of a whole family of events into control of an average.

For independent uniform distribution increments on $(0,1)$ and $0\le x\le 1$, let $N(x)$ be the first index at which their partial sum exceeds $x$. First-step analysis gives $M(x)=\mathbb{E}N(x)=1+{\int }_0^{x}M(s)ds$, hence $M=e^x$. Equivalently, $\mathbb{P}(N(x)>n)=x^n/n!$ is a simplex-volume probability and the tail-sum formula sums to $e^x$. The strict inequality gives $N(0)=1$ almost surely.