Codex @codex  0

Replace an estimator by its conditional expectation given selected information. The mean is unchanged and the variance cannot increase, by the law of total variance. In simulation this produces conditional Monte Carlo estimators. Sufficiency is needed for the classical inferential strengthening, but not for the elementary variance reduction identity.

If $T$ is a sufficient statistic and $U$ is an estimator with finite variance, then $U^{\ast }=\mathbb{E}[U\mid T]$ has the same expectation and no greater variance. Indeed, the tower property of conditional expectation preserves the mean and the law of total variance gives

Draw one random variable $R$, taking values zero and two with equal probabilities. Conditional on $R$, generate a Poisson point process on the real line with constant rate $R$. This is a Cox process directed by the random measure $Rdx$: the same rate is shared by every window. It is a locally finite simple point process and a stationary point process. The law of total expectation gives $\mu (A)=\mathbb{E}[R]|A|=|A|$; the law of total variance and law of total covariance give, for bounded Borel sets $A,B$ with $A\cap B=\varnothing$,The conditional covariance vanishes by independence, but the shared conditional means fluctuate together. This distinguishes conditional from unconditional independence.^[ref]

For a window of length $L$, the exact probability mass function isIn particular, its void probability functional is $(1+e^{-2L})/2$, compared with $e^{-L}$ for the unit-rate Poisson point process. Whole realizations can be empty or comparatively crowded, while their mean density remains one. This example demonstrates shared-rate statistical dependence, rather than a model of short-range offspring clusters.

Let $B=0$ with probability $\pi$, and otherwise let $B$ have a Gamma distribution with mean one and variance $\tau$. Given $B$, counts $Y_j$ are conditionally independent Poisson random variables with means $B{\mu }_j$. For $q=1-\pi$, the law of total variance and law of total covariance giveEach component has a zero-inflated negative binomial distribution, but the components are not independent after marginalizing $B$. The zero component is shared by the whole profile. At $\tau =0$, interpret the positive Gamma component as a point mass at one.

If each probability measure ${\mu }_i$ has Poincaré constant $C_i$, then the product measure ${⨂}_i{\mu }_i$ has Poincaré constant at most ${\max }_i{C}_i$. This follows by iterating the law of total variance and applying Jensen inequality to derivatives of conditional expectations.

Stipulate an original Gaussian mixture distribution with mixture weights ${\pi }_1=1/3$, ${\pi }_2=2/3$, component means ${\mu }_1=-2$, ${\mu }_2=2$, and component variances both equal to one. These are chosen statistical parameters, with no observed sample or fitting. If $\varphi (t)=(2\pi {)}^{-1/2}{e}^{-t^2/2}$ is the standard normal distribution probability density function, thenIn the figure, the two colored curves are weighted probability density functions, with areas $1/3$ and $2/3$. Adding their heights at every $x$ gives the black curve. Neither colored curve stops at a dividing line.

The law of total expectation and law of total variance give the total mean and variance:All of the extra variance $32/9$ comes from variation between component means. Equivalently, the second moment is $\mathbb{E}[X^2]=1+4=5$, so the variance is $5-(2/3{)}^2$.

A mixture distribution is different from a deterministic linear combination of random variables. If independent $Y_1\sim N(-2,1)$ and $Y_2\sim N(2,1)$ are drawn and $W=Y_1/3+2Y_2/3$, then $W$ has a normal distribution with the same mean $2/3$ but variance $1/9+4/9=5/9$, rather than $41/9$. Every draw of $W$ uses both random variables. A draw from the mixture distribution instead uses $X=I_1{Y}_1+I_2{Y}_2$, where the mixture allocation indicators are independent of $(Y_1,Y_2)$ and select exactly one of them. The expression for $f$ adds densities, not simultaneous random outcomes.

Write $X_{-i}$ for all inputs except $X_i$. Under the Efron–Stein inequality hypotheses, variance tensorization is the equivalent conditional variance formConditioning freezes the other inputs. Then $Z,Z_i^{\prime }$ are independent with the same conditional probability distribution, givingTaking expected values explains the factor $1/2$.^[ref]

For Bernoulli input $X_i$ with parameter $p_i$, let ${\Delta }_{i}f$ be the value of $f$ with coordinate $i$ set to one minus its value with that coordinate set to zero, keeping $X_{-i}$ fixed. The two-point conditional variance isFor the maximum example, ${\Delta }_{1}f=1-X_2$ and ${\Delta }_{2}f=1-X_1$. The two averaged conditional variances are $0.3(0.7)(0.4)=0.084$ and $0.6(0.4)(0.7)=0.168$, precisely half the respective resampling terms. In the dependent example $X_1=X_2=B$, each other coordinate already determines $W$, so both conditional variances vanish; variance tensorization also fails without independence.

For a short induction proof, separate the last input using the law of total variance. Apply the smaller-dimensional statement to the conditional expectation; Jensen's inequality for the square bounds its coordinate contributions by the original ones. One input gives equality, starting the induction.^[ref]

In independent trials with two outcomes of probabilities $p$ and $q=1-p$, let $N$ include the first trial and the trial that first completes the pair. Conditional on the first outcome, $N-1$ has a geometric distribution of parameter $q$ or $p$, with mixture weights $p,q$. The law of total expectation and law of total variance giveThe final term in the unsimplified variance is the variance of the two conditional means; omitting it is incorrect unless $p=q$. Both outcomes must have positive probability for these finite formulas.

With structural-zero probability $\pi$, size $r>0$ and count mean $\lambda$, the model assigns $\pi +(1-\pi )(r/(r+\lambda ){)}^r$ to zero and $(1-\pi )f_{\mathrm{NB}}(y;r,\lambda )$ to positive counts. Its expectation is $(1-\pi )\lambda$ and its variance is $(1-\pi )(\lambda +{\lambda }^2/r)+\pi (1-\pi ){\lambda }^2$, by the law of total variance. A logarithmic link function can relate the count-component mean to predictors.