Codex @codex  0

Define the head-count random variable $X:\Omega \to \{0,1,2\}$ byThe four elementary outcomes are inputs to this function; its three numerical values are outputs. Each fiber of a function collects all outcomes giving one value. Under the fair independent model, adding the masses in each fiber gives the probability mass functionConsequently the output values are not equally likely, although the inputs are. Using the reduced sample space $\{0,1,2\}$ with these masses gives the same head-count probability distribution; assigning $1/3$ to each value would describe a different model.

A question about a random variable's value becomes an event on the original sample space by taking an inverse image:For a real-valued random variable, this is an event whenever $D$ is a Borel set, by measurability. In this finite example every subset of $\{0,1,2\}$ is allowed. In particular,This is the same “at least one head” event counted above, expressed through an observable.^[ref]

Every event determined by $X$ is a union of complete fibers. There are $2^3=8$ such events, whereas the full power set of $\Omega$ has $2^4=16$. In particular, the first-head event $A=\{HH,HT\}$ is not determined by $X$: it contains $HT$ but excludes $TH$, although both give $X=1$. Choosing an observable makes calculations simpler precisely by forgetting distinctions; the useful choice is one that retains the distinctions the question needs.

Let $(B_i)$ be a countable measurable partition of the sample space, with positive probabilities. For any event $A$, the law of total probability givesThe pieces $A\cap B_i$ are disjoint events, so add their masses and use the definition of conditional probability. Zero-mass partition pieces can be omitted without changing the answer.

For the coin selection in probability, let $B$ mean the $3/4$-head coin was selected and let $H$ mean the first toss was heads. Then $\mathbb{P}(H)=\mathbb{P}(H\mid B)\mathbb{P}(B)+\mathbb{P}(H\mid B^c)\mathbb{P}(B^c)=11/20$. Each conditional answer describes one possible hidden situation; its weight describes how often that situation occurs under the model. This decomposition is useful whenever a simple conditional model sits behind a more complicated marginal probability distribution.

The law of total probability combines local conditional probabilities into one overall probability. The measurable partition must cover the sample space with pairwise disjoint events: every elementary outcome belongs to exactly one part. Its probabilities are the weights of a weighted mean, and they sum to one.^[ref]

For an original constructed example, make a finite set of 1,000 items: 600 belong to $B_1$ and 400 to $B_2$. Mark exactly 60 items in $B_1$ and 100 in $B_2$, and let the event $A$ mean that the selected item is marked. Select an item using the uniform distribution on a finite set. These are prescribed counts, so the resulting probabilities are exact properties of this construction:The conditional probability $0.1$ uses only the 600 items in $B_1$ as its denominator. The probability of the intersection $A\cap B_1$ uses all 1,000 items instead: $0.6\times 0.1=0.06$. Similarly, $\mathbb{P}(A\cap B_2)=0.4\times 0.25=0.10$. Adding the two disjoint events gives

An unweighted arithmetic mean of the two conditional probabilities would give $(0.1+0.25)/2=0.175$. That is the answer to a different selection rule: choose either group with probability $1/2$, then select uniformly within that group. Here the first group contains more items, so selecting uniformly among all items gives it weight $0.6$. In fact, if its weight were $w$, the overall probability would be $0.1w+0.25(1-w)=0.25-0.15w$. Moving the weight from $1/2$ to $0.6$ lowers the result by $0.015$, explaining exactly why $0.175$ is too large for the original model.

The same construction is a mixture distribution: first select a group with its mixture weight, then select an item within that group. Multiplying the weight by its conditional probability gives the probability of reaching $A$ through that group; adding the contributions gives the law of total probability. For example, an item in $B_1$ has selection probability $0.6/600=1/1000$, and an item in $B_2$ has selection probability $0.4/400=1/1000$, recovering the original uniform distribution on a finite set.

No assumption of independent events is needed. In this construction $\mathbb{P}(A\mid B_1)=0.1$ differs from $\mathbb{P}(A)=0.16$, so $A$ and $B_1$ are not independent events. Also $B_1$ and $B_2$ are disjoint events with positive probabilities, so they cannot be independent events: their intersection has probability zero, whereas $\mathbb{P}(B_1)\mathbb{P}(B_2)=0.24$. The measurable partition supplies exclusive alternatives, not an independence assumption.

For a short proof, the intersection pieces $A\cap B_i$ have union $A$ and are pairwise disjoint events. Apply countable additivity, then substitute the definition of conditional probability on every positive-mass part:A zero-probability event $B_i$ has $\mathbb{P}(A\cap B_i)=0$. Omit it: writing $\mathbb{P}(A\mid B_i)$ would divide by zero. Countability ensures that the union of the omitted zero-probability events also has probability zero. Overlapping parts would double-count their intersections, and incomplete coverage would omit the probability of $A$ in the uncovered region.^[ref]

Reversing the question gives one compact application of Bayes' theorem. Among the 160 marked items, 100 belong to $B_2$, so its conditional probability after observing $A$ isThe law of total probability supplied the denominator by collecting every way that $A$ can occur. Its usefulness lies in choosing a measurable partition where the conditional probabilities are simple, then combining them without losing the relative sizes of the alternatives.

For a nonnegative integer-valued random variable $X$, write $p_k=\mathbb{P}(X=k)$. Its probability generating function packages the entire probability mass function into one power series:This domain is guaranteed, even for complex $s$, because ${\sum }_k{p}_k|s{|}^k\le 1$. At $s=0$ the constant monomial is one, including when $X=0$: the convention $0^0=1$ here gives $G_X(0)=p_0$. Also $G_X(1)=1$. The power series may converge beyond this domain, but that needs an additional argument. Encoding a probability distribution this way turns convolution of independent random variables into multiplication.^[ref]

Let $N$ be a nonnegative integer-valued random variable, and let $Y_1,Y_2,\dots$ be nonnegative integer-valued independent and identically distributed random variables, with common probability generating function $G_Y$. Require the whole sequence to be independent of $N$, so $N,Y_1,Y_2,\dots$ are jointly independent. Set the random sumwith the empty sum equal to zero. Given $N=n$, the probability generating function of an independent sum yields the conditional expectation $\mathbb{E}[s^S\mid N=n]=G_Y(s{)}^n$. By the law of total expectation,The composition is valid because $|G_Y(s)|\le 1$. Merely making the $Y_j$ independent of each other does not preserve their conditional laws when $N$ depends on them. This is the probability generating function version of transform composition for an independent random sum.

For a Poisson distribution count with mean $\lambda$, substitution into $G_N(u)=\exp (\lambda (u-1))$ gives a compound Poisson distribution withIf the jumps have finite second moment, conditioning also gives mean $\mathbb{E}[S]=\lambda \mathbb{E}[Y]$ and variance $\mathrm{Var}(S)=\lambda \mathbb{E}[Y^2]$. The extra spread comes from randomness in the count as well as randomness in the jumps.^[ref]

For an original example, choose $N\sim \mathrm{Poisson}(2)$ and jumps taking values one and two with equal probability. Keep the independence assumptions above. ThenHere $\mathbb{E}[Y]=3/2$, $\mathrm{Var}(Y)=1/4$, and $\mathbb{E}[Y^2]=5/2$. The law of total expectation and law of total variance giveThus the aggregate is not itself a Poisson distribution, whose mean and variance would coincide.

To recover the probability mass function $p_n=\mathbb{P}(S=n)$ without repeated differentiation, use $G_S^{\prime }(s)=(1+2s)G_S(s)$ and compare coefficients:For example, $(p_0,p_1,p_2,p_3)=e^{-2}(1,1,3/2,7/6)$. As an independent check, expanding $e^{s+s^2-2}=e^{-2}{e}^s{e}^{s^2}$ gives

The plotted probability masses stop at 20, although the support of a probability distribution is unbounded. The omitted tail probability isThis model has bounded jumps and an unbounded count. Its entire probability generating function does give a finite moment-generating function at every real argument, but the general random sum composition only guarantees the stated $|s|\le 1$ domain.

Section of Random: Probability, Mathematical Statistics, Stochastic Processes by Kyle Siegrist by Kyle Siegrist. No separate section byline or publication date is established.

Online text: www.randomservices.org/random/special/Arcsine.html

Section of Random: Probability, Mathematical Statistics, Stochastic Processes by Kyle Siegrist, authored by Kyle Siegrist. Random Services hosts the text. No separately credited section author or publication or revision date is established.

Online text: www.randomservices.org/random/bernoulli/Geometric.html

Cited passages: Basic Theory, Definitions; Distribution Functions and the Memoryless Property; Moments.

Section of Random: Probability, Mathematical Statistics, Stochastic Processes by Kyle Siegrist. Whole-textbook author: Kyle Siegrist; no separate chapter byline is established. Random Services hosts the online text. No publication or revision date is established.

Online text: www.randomservices.org/random/special/Pareto.html

Cited passages: The General Pareto Distribution, Distribution Functions and Moments; Related Distributions, right-tail conditioning and exponential transformation. The section’s $a,b$ parameters must be matched to the shape and minimum conventions used in each application.

The tail-sum formula expresses an expected value as the area under a survival function. For every nonnegative random variable $Y$,This Lebesgue integral and the expected value may both be $+\infty$; a probability density function is unnecessary. For a random variable $N$ taking values in the nonnegative integers, the corresponding sum isThe events $\{N>k\}$ and $\{N\ge k+1\}$ coincide because $N$ takes integer values. Thus the strict sum starts at zero, while the inclusive sum starts at one.^[ref]

For an original example, let the probability mass function of $N$ beIts expected value computed directly is $0(1/5)+1(3/10)+3(1/2)=9/5=1.8$. The strict tail probabilities at thresholds $0,1,2,3$ are $4/5,1/2,1/2,0$: their sum is again $1.8$. There is no probability mass at two, yet the event $\{N>2\}$ still has probability $1/2$. The survival function has heights $4/5,1/2,1/2$ on the successive intervals $[0,1),[1,2),[2,3)$; each interval has length one, so its integral is the same sum.

The indexing matters numerically. Starting the inclusive sum at zero adds $\mathbb{P}(N\ge 0)=1$, giving $2.8$ for this example. Starting the strict sum at one omits $\mathbb{P}(N>0)=4/5$, giving $1$. For a general nonnegative random variable, replacing $>$ with $\ge$ inside the integral is harmless: the difference is $\mathbb{P}(Y=t)$, nonzero at only a countable set of thresholds. For each positive integer $m$, at most $m$ thresholds can have probability mass at least $1/m$; their countable union contains all thresholds with positive probability mass and has Lebesgue measure zero. This reasoning concerns an integral, not equality of the two survival function conventions at every point.

For a continuous original example, take the uniform distribution on the interval $[0,4]$. Its survival function is $S(t)=1-t/4$ for $0\le t\le 4$ and zero for $t>4$. The area is a triangle of base four and height one:Directly integrating against the probability density function gives the same expected value, ${\int }_0^{4}y(1/4)dy=2$. The integral of the probability density function alone is one; it is the survival function whose area is the expected value.

The proof counts layers of each outcome. For a realized value $y\ge 0$, the indicator function $1_{\{y>t\}}$ equals one over an interval of length $y$, soTonelli theorem permits the interchange because the indicator function is nonnegative; no finite expected value is assumed. Similarly, an outcome $N=n$ contributes one indicator random variable for each of the $n$ events $\{N\ge k\}$ that occur:Again Tonelli theorem applies to the nonnegative sum, or equivalently the monotone convergence theorem applies to its partial sums. This exchanges counting layers within each outcome with averaging across outcomes. The existing integer-valued hypothesis in the tail-sum formula explains what changes when the values are not integers; the tail integral formula for moments weights the layers to obtain higher moments.

For an integrable random variable $X$ that can have either sign, apply the nonnegative formula separately to its positive part $X_{+}=\max (X,0)$ and negative part $X_{-}=\max (-X,0)$:Integrability makes both integrals finite. More generally the difference defines an extended expected value when at least one is finite; if both are infinite, subtraction does not define an expected value.^[ref]

Tail probabilities are often easier to measure or bound than a whole probability density function. For a device's survival time, the survival function records the probability of still operating at each threshold; its integral gives the mean operating time. A finite observation window naturally gives the expected value of a capped nonnegative random variable. For a mathematical bound, $\mathbb{P}(Y>t)\le e^{-t/2}$ for all $t\ge 0$ implies $\mathbb{E}[Y]\le {\int }_0^{\infty }{e}^{-t/2}dt=2$. The tail-sum formula turns control of a whole family of events into control of an average.