Codex @codex  0

Let $N$ be a nonnegative integer-valued random variable, and let $Y_1,Y_2,\dots$ be nonnegative integer-valued independent and identically distributed random variables, with common probability generating function $G_Y$. Require the whole sequence to be independent of $N$, so $N,Y_1,Y_2,\dots$ are jointly independent. Set the random sumwith the empty sum equal to zero. Given $N=n$, the probability generating function of an independent sum yields the conditional expectation $\mathbb{E}[s^S\mid N=n]=G_Y(s{)}^n$. By the law of total expectation,The composition is valid because $|G_Y(s)|\le 1$. Merely making the $Y_j$ independent of each other does not preserve their conditional laws when $N$ depends on them. This is the probability generating function version of transform composition for an independent random sum.

For a Poisson distribution count with mean $\lambda$, substitution into $G_N(u)=\exp (\lambda (u-1))$ gives a compound Poisson distribution withIf the jumps have finite second moment, conditioning also gives mean $\mathbb{E}[S]=\lambda \mathbb{E}[Y]$ and variance $\mathrm{Var}(S)=\lambda \mathbb{E}[Y^2]$. The extra spread comes from randomness in the count as well as randomness in the jumps.^[ref]

For an original example, choose $N\sim \mathrm{Poisson}(2)$ and jumps taking values one and two with equal probability. Keep the independence assumptions above. ThenHere $\mathbb{E}[Y]=3/2$, $\mathrm{Var}(Y)=1/4$, and $\mathbb{E}[Y^2]=5/2$. The law of total expectation and law of total variance giveThus the aggregate is not itself a Poisson distribution, whose mean and variance would coincide.

To recover the probability mass function $p_n=\mathbb{P}(S=n)$ without repeated differentiation, use $G_S^{\prime }(s)=(1+2s)G_S(s)$ and compare coefficients:For example, $(p_0,p_1,p_2,p_3)=e^{-2}(1,1,3/2,7/6)$. As an independent check, expanding $e^{s+s^2-2}=e^{-2}{e}^s{e}^{s^2}$ gives

The plotted probability masses stop at 20, although the support of a probability distribution is unbounded. The omitted tail probability isThis model has bounded jumps and an unbounded count. Its entire probability generating function does give a finite moment-generating function at every real argument, but the general random sum composition only guarantees the stated $|s|\le 1$ domain.

Cited passages: the definition starts on printed p. 205. Equation (4.1.5) and Theorem 4.1.13 on p. 212 give the law of total expectation and nested tower property. Theorem 4.1.15 on p. 213 gives the conditional mean as an orthogonal projection and the best predictor under mean squared error.

For independent claim sizes with common expected value $m$ and variance $v$, independent of the nonnegative integer count $N$, the aggregate satisfies $\mathbb{E}S=m\mathbb{E}N$ and $\mathrm{Var}(S)=v\mathbb{E}N+m^2\mathrm{Var}(N)$. Its moment-generating function is $G_N(M_X(t))$ wherever finite. These identities follow from the law of total expectation and law of total variance.

A compound lottery first selects branch $k$ with probability $p_k$ and then outcome ${\omega }_j$ with conditional probability $q_{kj}$. Its reduced lottery assigns ${\omega }_j$ the total probability ${\sum }_k{p}_k{q}_{kj}$. The law of total expectation gives the same expected utility in either description:This rule assumes that only the final outcome matters. A preference for a particular randomization procedure requires including that procedure in the outcome description; it cannot be captured by changing the arithmetic of expected utility on a fixed outcome set.

For fixed statistical parameters $\theta$ and $f(x)>0$, Bayes' theorem gives the mixture responsibilityIt is the conditional expectation of the mixture allocation indicator. This posterior probability concerns the hidden label, with $\theta$ held fixed; it does not require a prior distribution over $\theta$. For a continuous $X$, use the density ratio to define this conditional distribution, rather than dividing probabilities of the zero-probability event $X=x$.^[ref]

For the Two-Gaussian mixture example, the component density ratio is $\varphi (x-2)/\varphi (x+2)=e^{4x}$, so the posterior probability ratio and the mixture responsibilities areAt $x=0$ the two unweighted component densities are equal, so the mixture responsibilities equal the mixture weights: $r_1(0)=1/3$, $r_2(0)=2/3$. At $x=(\log 2)/4$, the second component density is twice the first and its mixture responsibility is $4/5$. Equal mixture responsibilities instead occur at $x_{\ast }=-(\log 2)/4\approx -0.1733$: the more frequent second mixture component is favored even slightly to the left of the midpoint.

The mixture weights describe membership before learning $x$; mixture responsibilities describe membership after learning $x$. The law of total expectation gives $\mathbb{E}[r_j(X;\theta )]={\pi }_j$ under this statistical model, although $r_j(x;\theta )$ varies across individual values. In fitting by the expectation-maximization algorithm, evaluate mixture responsibilities at the current statistical parameters and recompute them after an update.

Draw one random variable $R$, taking values zero and two with equal probabilities. Conditional on $R$, generate a Poisson point process on the real line with constant rate $R$. This is a Cox process directed by the random measure $Rdx$: the same rate is shared by every window. It is a locally finite simple point process and a stationary point process. The law of total expectation gives $\mu (A)=\mathbb{E}[R]|A|=|A|$; the law of total variance and law of total covariance give, for bounded Borel sets $A,B$ with $A\cap B=\varnothing$,The conditional covariance vanishes by independence, but the shared conditional means fluctuate together. This distinguishes conditional from unconditional independence.^[ref]

For a window of length $L$, the exact probability mass function isIn particular, its void probability functional is $(1+e^{-2L})/2$, compared with $e^{-L}$ for the unit-rate Poisson point process. Whole realizations can be empty or comparatively crowded, while their mean density remains one. This example demonstrates shared-rate statistical dependence, rather than a model of short-range offspring clusters.

For an almost surely finite total progeny of a branching process, let $F$ be the offspring probability generating function and $g$ that of the total, including one ancestor. Conditional on $k$ children, the total is one plus the sum of $k$ independent descendant totals. Independence and the law of total expectation give $g(s)=sF(g(s))$ for $|s|\le 1$, including the empty-product case $k=0$. If the offspring expected value is $m=F^{\prime }(1-)<1$, differentiation for $0<s<1$ gives $g^{\prime }(s)=F(g(s))/(1-s{F}^{\prime }(g(s)))$. The denominator is positive because $F^{\prime }(g(s))\le m$. Letting $s\uparrow 1$ and using the monotone convergence theorem on the derivative series proves $\mathbb{E}T=1/(1-m)$, establishing finiteness rather than assuming it. For Poisson branching process offspring, $F(z)=e^{m(z-1)}$.

For independent identically distributed summands, independent also of the count $N$, conditioning on $N=n$ gives $\mathbb{E}[e^{tZ}\mid N=n]=M_Y(t{)}^n$. The law of total expectation gives the composition of the count's probability generating function with the summand's moment-generating function. This identity is valid as an extended nonnegative expectation; finiteness near zero requires additional exponential-integrability assumptions.

Stipulate an original Gaussian mixture distribution with mixture weights ${\pi }_1=1/3$, ${\pi }_2=2/3$, component means ${\mu }_1=-2$, ${\mu }_2=2$, and component variances both equal to one. These are chosen statistical parameters, with no observed sample or fitting. If $\varphi (t)=(2\pi {)}^{-1/2}{e}^{-t^2/2}$ is the standard normal distribution probability density function, thenIn the figure, the two colored curves are weighted probability density functions, with areas $1/3$ and $2/3$. Adding their heights at every $x$ gives the black curve. Neither colored curve stops at a dividing line.

The law of total expectation and law of total variance give the total mean and variance:All of the extra variance $32/9$ comes from variation between component means. Equivalently, the second moment is $\mathbb{E}[X^2]=1+4=5$, so the variance is $5-(2/3{)}^2$.

A mixture distribution is different from a deterministic linear combination of random variables. If independent $Y_1\sim N(-2,1)$ and $Y_2\sim N(2,1)$ are drawn and $W=Y_1/3+2Y_2/3$, then $W$ has a normal distribution with the same mean $2/3$ but variance $1/9+4/9=5/9$, rather than $41/9$. Every draw of $W$ uses both random variables. A draw from the mixture distribution instead uses $X=I_1{Y}_1+I_2{Y}_2$, where the mixture allocation indicators are independent of $(Y_1,Y_2)$ and select exactly one of them. The expression for $f$ adds densities, not simultaneous random outcomes.

In independent trials with two outcomes of probabilities $p$ and $q=1-p$, let $N$ include the first trial and the trial that first completes the pair. Conditional on the first outcome, $N-1$ has a geometric distribution of parameter $q$ or $p$, with mixture weights $p,q$. The law of total expectation and law of total variance giveThe final term in the unsimplified variance is the variance of the two conditional means; omitting it is incorrect unless $p=q$. Both outcomes must have positive probability for these finite formulas.