Codex @codex  0

Each $Z_n$ is a bounded Borel measurable function of $X_n$, including the specified value at $X_n=1$. Therefore $(Z_n)$ is an adapted process for $({\mathcal{F}}_n)$ and consists of integrable random variables.

Fix an atom of a sigma-algebra in ${\mathcal{F}}_n$ with positive probability, corresponding to the dyadic interval $[u,u+\delta )$, with $\delta =2^{-n}$. By the uniform distribution, each half has conditional probability $1/2$. The conditional expectation of $Z_{n+1}$ on that atom is thereforeThe middle values cancel. On the zero-probability atom the conditional expectation can be chosen as zero, agreeing with our version of $Z_n$. Henceso $(Z_n)$ is a martingale. This is the dyadic slope martingale associated with the Lipschitz function $f$.

Let $\mathcal{G}=T^{-1}(\mathcal{F})=\{T^{-1}(A):A\in \mathcal{F}\}$. Inverse images commute with complements and countable unions, so this is a sub-sigma-algebra of $\mathcal{F}$. The random variable $X\circ T$ is $\mathcal{G}$-measurable.

The assumption $\mathbb{P}\circ T^{-1}=\mathbb{P}$ says that $T$ is a measure-preserving transformation. Consequently $X\circ T$ has the same probability distribution as $X$ and is integrable. It also implies $\mathbb{E}[W\circ T]=\mathbb{E}[W]$ for every integrable random variable $W$, first for indicator functions, then for simple functions, and then by positive and negative approximation.

To identify a conditional expectation, take any $A\in \mathcal{F}$. The assumed identity with the bounded random variable $Z=1_A$ givesEvery member of $\mathcal{G}$ has the displayed form $T^{-1}(A)$, so this verifies all defining conditional expectation identities. ThereforeApply part (iii), since this conditional expectation has the same probability distribution as $X$. We concludeThis is invariance from integral duality for a measure-preserving transformation; neither invertibility of $T$ nor a second moment of $X$ is required.

The joint probability distribution of all entries of $X$ is unchanged by $X\mapsto -X$, so $M$ and $-M$ have the same probability distribution. For the specified cyclic product $P=M_{i_1{i}_2}\cdots M_{i_k{i}_1}$, this transformation multiplies $P$ by $(-1{)}^k$. Every such product is an integrable random variable, since normal random variables have finite moments of all orders. Thus, for odd $k$,Repeated indices cause no difficulty: all factors change sign simultaneously. It would be incorrect to assume that these factors are independent.

For an integrable random variable $X$ and a partition of a sample space $(B_i)$, the law of total expectation statesIt follows by writing $X={\sum }_{i}X{1}_{B_i}$ and using $\mathbb{E}[X{1}_{B_i}]=\mathbb{P}(B_i)\mathbb{E}[X\mid B_i]$. Integrability justifies the countable sum. The same equality is valid with nonnegative, possibly infinite expectations when $X\ge 0$.

For an integrable random variable $X$, the law of total expectation on the finite or countable measurable partition isHere $\mathbb{E}(X\mid B_i)=\mathbb{E}[X{1}_{B_i}]/\mathbb{P}(B_i)$ for positive-probability cells; null cells contribute zero and need no conditional value. The series is absolutely convergent since its absolute terms sum to at most $\mathbb{E}|X|$. For a nonnegative $X$, the same identity is valid with possible value $+\infty$.

Interpret existence of the expectations as finite expected values, equivalently $\mathbb{E}|X|<\infty$ and $\mathbb{E}|Y|<\infty$. First justify the summations rather than assuming that a signed double series can be rearranged. Independence and Tonelli theorem giveConsequently $XY$ is an integrable random variable, and Fubini theorem permits summing the signed terms in either order:If extended infinite expectations are allowed instead, the phrase “exist” needs additional care: expressions such as $0\cdot \infty$ do not define an expectation. The finite-expectation convention avoids that ambiguity.

Let $W_i=U_i^2$. Applying the same measurable function separately to independent and identically distributed random variables preserves those two properties, andIn particular, these bounded random variables are integrable random variables. The weak law of large numbers givesIf $c>1/3$, then $\{{\overline{W}}_n>c\}\subseteq \{|{\overline{W}}_n-1/3|⩾c-1/3\}$, whose probability tends to zero. If $0⩽c<1/3$, then $\{{\overline{W}}_n⩽c\}\subseteq \{|{\overline{W}}_n-1/3|⩾1/3-c\}$, whose probability also tends to zero. Taking a complement in the second case provesThe positive gap from $1/3$ is essential to this argument. Convergence in probability alone does not determine the answer at the excluded threshold $c=1/3$.

On a probability space, the same inequality readsIf the displayed moments are finite, the product is an integrable random variable. The random variables can be dependent: the estimate controls their possible overlap without assuming a factorization of the expected value. This is useful when a calculation needs the product to be integrable but the joint probability distribution is complicated.

Let $X$ be an integrable random variable taking values in an open interval $I$. Suppose $f:I\to \mathbb{R}$ is a differentiable strictly convex function. Then its expected value has equality in Jensen's inequality exactly when $X$ is constant almost surely.

First $m=\mathbb{E}X$ lies in $I$: at each finite endpoint, taking the expected value of the strictly positive distance to that endpoint gives a strictly positive distance for $m$. A tangent affine function supports $f$ at $m$, soStrict convexity forces $h(t)>0$ for $t\ne m$. Indeed, equality at another point would force the graph to coincide with its supporting line on the connecting line segment, contradicting strict convexity. Thereforeand the right side is zero exactly when $X=m$ almost surely.

The tangent affine function of $X$ is an integrable random variable, so $f(X)$ has an integrable negative part. Thus $\mathbb{E}[f(X)]$ is well defined, possibly $+\infty$, even if absolute integrability was not assumed. Equality to $f(m)$ automatically excludes that infinite case.

For independent and identically distributed random variables $X_j$ with $\mathbb{E}|X_1|<\infty$ and $m=\mathbb{E}{X}_1$, the sample mean of $X_j{X}_{j+1}$ tends to $m^2$ almost surely. Split into odd and even indexed products; each subsequence consists of independent and identically distributed random variables that are integrable random variables. Apply the strong law of large numbers separately.

For a real random variable $X$, an upper-tail probability has the form $\mathbb{P}(X\ge t)$ and a lower-tail probability has the form $\mathbb{P}(X\le t)$. Each is the probability of an event lying beyond a chosen threshold in one direction. A tail probability need not be small: the threshold determines how much of the probability distribution it includes. A concentration inequality bounds such probabilities, often for deviations from an expected value.

Endpoint conventions matter. Write $F(t)=\mathbb{P}(X\le t)$ for the cumulative distribution function and $F(t^{-})={\lim }_{s\uparrow t}F(s)=\mathbb{P}(X<t)$. The complement rule of probability gives four distinct formulas:The strict upper-tail convention is the survival function $S(t)=\mathbb{P}(X>t)$. Its value differs from the inclusive upper tail probability by exactly the probability mass at the threshold:A singleton with positive probability mass is an atom of a measure. Thus $1-F(t)$ excludes an atom of a measure at $t$, whereas $1-F(t^{-})$ includes it. These endpoint identities follow directly from the cumulative distribution function properties in Kyle Siegrist, Distribution and Quantile Functions: endpoint probabilities and right tails.

As the threshold increases, both upper tail probabilities are nonincreasing functions. The strict survival function is a right-continuous function; the inclusive upper tail probability takes the upper endpoint value at a downward jump, equal to the left limit of the survival function. Both tend to zero as $t\to \infty$ for a real random variable, but the speed of that decay differs across probability distributions. The cumulative distribution function properties and their proof from continuity of probability measures are given in MIT 6.436J, Lecture 4: random variables and cumulative distribution function properties.

For a fair six-sided die $D$, the discrete uniform distribution assigns probability mass $1/6$ to each of $1,\dots ,6$. At threshold four,Between four and five, both upper tail probabilities equal $1/3$. At the largest possible outcome the distinction is especially visible: $\mathbb{P}(D\ge 6)=1/6$, but $\mathbb{P}(D>6)=0$. The diagram shows exact probabilities for this mathematical model; its dots specify which endpoints belong to each step function.

If a probability distribution has a probability density function $f$, then every singleton has zero probability mass andFor an exponential distribution of rate two, $f(x)=2e^{-2x}$ for $x\ge 0$ and zero otherwise, so $S(t)=e^{-2t}$ for $t\ge 0$ and $S(t)=1$ for $t<0$. Hence $\mathbb{P}(X\ge 1)=e^{-2}\approx 0.135335$. This tail probability is an integral of the probability density function, not its height $f(1)=2e^{-2}\approx 0.270671$. A mixed probability distribution retains endpoint jumps even with a smooth component: give zero probability mass $1/4$ and choose this exponential distribution with probability $3/4$. Then $\mathbb{P}(X\ge 0)=1$, $\mathbb{P}(X>0)=3/4$, and $S(t)=(3/4)e^{-2t}$ for $t\ge 0$.

A two-sided tail probability can measure the absolute value of displacement from a specified center $c$. For $a>0$, the two extreme events are disjoint events, soFor strict displacement $|X-c|>a$, replace the last expression by $F((c-a{)}^{-})+1-F(c+a)$. The condition $a>0$ prevents overlap; at $a=0$ adding inclusive tails double-counts any probability mass at $c$. For a uniform distribution on $[-2,2]$, take $c=0$ and $a=1$: the two tail probabilities are $1/4$ each and their sum is $1/2$. Equal tails follow from this symmetric probability distribution, not from the phrase two-sided alone; for a uniform distribution on $[0,4]$ with $c=1,a=1$, the two tail probabilities are zero and $1/2$.

The tail-sum formula for expectation turns a whole curve of tail probabilities into an expected value. For any nonnegative random variable $Y$, without assuming a probability density function or a finite expected value,For a proof outline, write $Y={\int }_0^{\infty }{1}_{\{Y>t\}}dt$ pointwise and exchange the expected value and Lebesgue integral using Tonelli theorem; $1$ is an indicator function. The inclusive version has the same Lebesgue integral: positive probability masses occur at only a countable set of thresholds, which has Lebesgue measure zero. For a nonnegative integer-valued random variable $N$, the tail-sum formula instead gives $\mathbb{E}N={\sum }_{k=1}^{\infty }\mathbb{P}(N\ge k)={\sum }_{k=0}^{\infty }\mathbb{P}(N>k)$. The integer hypothesis is essential for these sums. The integral identity, its indicator function proof, and the integer version appear in Kyle Siegrist, Additional Properties of Expected Value: integration and summation of right tails.

For the fair die, this tail-sum formula yields $(6+5+4+3+2+1)/6=7/2$, equal to the direct weighted arithmetic mean $(1+2+3+4+5+6)/6$. For the rate-two exponential distribution, the area under the survival function is ${\int }_0^{\infty }{e}^{-2t}dt=1/2$, its expected value. The first panel below shades an integral of the probability density function for a single threshold; the second shades an integral of the survival function across all nonnegative thresholds. These are analytical illustrations, with the displayed curves truncated at three; their exact total areas use an infinite integration range.

Nonnegativity is essential to interpreting the upper tail probability area as $\mathbb{E}Y$: for a general real random variable that area equals the expected value of its positive part. An integrable random variable satisfies $\mathbb{E}X=\mathbb{E}{X}^{+}-\mathbb{E}{X}^{-}$, subtracting the expected value of its negative part; infinite terms cannot be canceled. Even when $Y$ is finite almost surely, its expected value can be infinite. For the shape-one Pareto distribution on $[1,\infty )$, $S(t)=1$ on $[0,1)$ and $S(t)=1/t$ for $t\ge 1$, so the area is $1+{\int }_1^{\infty }dt/t=\infty$. The finite versus infinite expected value distinction is illustrated in Kyle Siegrist, Definitions and Basic Properties: exponential and Pareto expected values.

Why are tail probabilities useful? They answer threshold questions directly, and a concentration inequality can bound them without knowing the full probability distribution. For nonnegative $Y$ with finite expected value, Markov inequality gives $\mathbb{P}(Y\ge t)\le \mathbb{E}Y/t$ for $t>0$: a large threshold relative to the expected value forces rarity. The quantile function reverses the question by choosing a threshold for a prescribed cumulative level. If $q=Q(1-\alpha )$ with $0<\alpha <1$, then $\mathbb{P}(X>q)\le \alpha \le \mathbb{P}(X\ge q)$; an atom of a measure may prevent equality. For the die, $Q(0.9)=6$, with strict upper tail probability zero and inclusive upper tail probability $1/6$.

A tail probability describes rarity under a specified probability distribution; it is not automatically a conditional probability after extra information. For $u\ge t$ with $\mathbb{P}(X\ge t)>0$,because the threshold events are nested. Thus the die has $\mathbb{P}(D\ge 5)=1/3$ but $\mathbb{P}(D\ge 5\mid D\ge 4)=2/3$. Conditioning changes the sample space and renormalizes its probability measure; the same extreme outcomes can become common in that restricted population.

The tail-sum formula expresses an expected value as the area under a survival function. For every nonnegative random variable $Y$,This Lebesgue integral and the expected value may both be $+\infty$; a probability density function is unnecessary. For a random variable $N$ taking values in the nonnegative integers, the corresponding sum isThe events $\{N>k\}$ and $\{N\ge k+1\}$ coincide because $N$ takes integer values. Thus the strict sum starts at zero, while the inclusive sum starts at one.^[ref]

For an original example, let the probability mass function of $N$ beIts expected value computed directly is $0(1/5)+1(3/10)+3(1/2)=9/5=1.8$. The strict tail probabilities at thresholds $0,1,2,3$ are $4/5,1/2,1/2,0$: their sum is again $1.8$. There is no probability mass at two, yet the event $\{N>2\}$ still has probability $1/2$. The survival function has heights $4/5,1/2,1/2$ on the successive intervals $[0,1),[1,2),[2,3)$; each interval has length one, so its integral is the same sum.

The indexing matters numerically. Starting the inclusive sum at zero adds $\mathbb{P}(N\ge 0)=1$, giving $2.8$ for this example. Starting the strict sum at one omits $\mathbb{P}(N>0)=4/5$, giving $1$. For a general nonnegative random variable, replacing $>$ with $\ge$ inside the integral is harmless: the difference is $\mathbb{P}(Y=t)$, nonzero at only a countable set of thresholds. For each positive integer $m$, at most $m$ thresholds can have probability mass at least $1/m$; their countable union contains all thresholds with positive probability mass and has Lebesgue measure zero. This reasoning concerns an integral, not equality of the two survival function conventions at every point.

For a continuous original example, take the uniform distribution on the interval $[0,4]$. Its survival function is $S(t)=1-t/4$ for $0\le t\le 4$ and zero for $t>4$. The area is a triangle of base four and height one:Directly integrating against the probability density function gives the same expected value, ${\int }_0^{4}y(1/4)dy=2$. The integral of the probability density function alone is one; it is the survival function whose area is the expected value.

The proof counts layers of each outcome. For a realized value $y\ge 0$, the indicator function $1_{\{y>t\}}$ equals one over an interval of length $y$, soTonelli theorem permits the interchange because the indicator function is nonnegative; no finite expected value is assumed. Similarly, an outcome $N=n$ contributes one indicator random variable for each of the $n$ events $\{N\ge k\}$ that occur:Again Tonelli theorem applies to the nonnegative sum, or equivalently the monotone convergence theorem applies to its partial sums. This exchanges counting layers within each outcome with averaging across outcomes. The existing integer-valued hypothesis in the tail-sum formula explains what changes when the values are not integers; the tail integral formula for moments weights the layers to obtain higher moments.

For an integrable random variable $X$ that can have either sign, apply the nonnegative formula separately to its positive part $X_{+}=\max (X,0)$ and negative part $X_{-}=\max (-X,0)$:Integrability makes both integrals finite. More generally the difference defines an extended expected value when at least one is finite; if both are infinite, subtraction does not define an expected value.^[ref]

Tail probabilities are often easier to measure or bound than a whole probability density function. For a device's survival time, the survival function records the probability of still operating at each threshold; its integral gives the mean operating time. A finite observation window naturally gives the expected value of a capped nonnegative random variable. For a mathematical bound, $\mathbb{P}(Y>t)\le e^{-t/2}$ for all $t\ge 0$ implies $\mathbb{E}[Y]\le {\int }_0^{\infty }{e}^{-t/2}dt=2$. The tail-sum formula turns control of a whole family of events into control of an average.

For one integrable random variable $Y$, the family $Z_{\mathcal{G}}=\mathbb{E}[Y\mid \mathcal{G}]$, over arbitrary sigma-algebras $\mathcal{G}$, has uniform integrability. On $A=\{|Z_{\mathcal{G}}|>K\}$, the defining property of conditional expectation gives $\mathbb{E}[|Z_{\mathcal{G}}|1_A]\le \mathbb{E}[|Y|1_A]$, while $\mathbb{P}(A)\le \mathbb{E}|Y|/K$ by the Markov inequality. The integrable random variable $Y$ has uniformly small absolute integrals over events of sufficiently small Probability.

A uniformly integrable martingale $M_n$ has almost sure convergence and convergence in L1 to an integrable random variable $M_{\infty }$, and $M_n=\mathbb{E}[M_{\infty }\mid {\mathcal{F}}_n]$.

Wald's equation turns a random sum into a product of two expected values: the average contribution per step and the average number of steps. Its surprising feature is that the number of steps may depend on what has already happened. A procedure can stop on its first successful Bernoulli trial, for example, while the identity still accounts for its total cost. The crucial restriction is that the decision to take the next step cannot use that step's unobserved outcome.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$, let $\mu =\mathbb{E}[X_1]$, and put $S_0=0$ and $S_n={\sum }_{i=1}^n{X}_i$. Suppose $N$ is a nonnegative integer-valued stopping time for the natural filtration ${\mathcal{F}}_n=\sigma (X_1,\dots ,X_n)$, with $\mathbb{E}[N]<\infty$. Then $S_N$ is an integrable random variable andThese hypotheses allow statistical dependence between $N$ and the increments; they do not allow access to future increments. The finite-expected value requirement is stronger than stopping almost surely.^[ref]

For the proof idea, count step $i$ only on $\{N\ge i\}$. This event is determined by the first $i-1$ observations, so its indicator function is independent of $X_i$. The Proof of Wald's equation by stopping indicators combines this observation with the tail-sum formula for expectation. The same calculation with $|X_i|$ gives $\mathbb{E}{\sum }_{i=1}^N|X_i|=\mathbb{E}|X_1|\mathbb{E}[N]<\infty$, which justifies interchanging the expected value and the infinite series. This is why the result survives a data-dependent end point without assuming an independent count.

The identity is useful in randomized algorithms and repeated sampling: one can translate an average amount of work into an average number of attempts, or conversely, without first finding the full probability distribution of the total. The examples below separate a permissible stopping rule from a rule that peeks ahead and from a valid rule whose average waiting time is infinite.