Codex @codex  0

The random variables are independent random variables if the events determined by one are independent of the events determined by the other. For integer-valued variables, this is equivalent to the factorizationIndeed, independence gives the factorization by taking singleton events. Conversely, summing it over $x\in A$ and $y\in B$, for any subsets $A,B\subseteq \mathbb{Z}$, gives $\mathbb{P}(X\in A,Y\in B)=\mathbb{P}(X\in A)\mathbb{P}(Y\in B)$. Thus the singleton condition implies independence of all events in question.

Under the $p$-biased product measure ${\mu }_p$ on $\{-1,1{\}}^n$, the coordinates are independent random variables with $\mathbb{P}(x_i=-1)=p$ and $\mathbb{P}(x_i=1)=1-p$. Writing $\mu =1-2p$, $\sigma =2\sqrt{p(1-p)}$, and ${\varphi }_i=(x_i-\mu )/\sigma$, the products ${\varphi }_S={\prod }_{i\in S}{\varphi }_i$ form the $p$-biased Fourier basis.

An encryption scheme has perfect secrecy when the ciphertext and plaintext are independent random variables, so observing the ciphertext does not change the plaintext's probability distribution.

Let $X_1,\dots ,X_n$ be independent random variables, each choosing a label from $\{1,\dots ,n\}$. Put their probabilities in the matrix $P=(p_{ij})$, where $p_{ij}=\mathbb{P}(X_i=j)$. Each row sums to one. All choices are distinct precisely when their ordered list is one of the permutations of the $n$ labels. These are mutually exclusive events, and independence makes the probability of one list $\sigma$ equal to ${\prod }_i{p}_{i,\sigma (i)}$. Adding the list probabilities gives the permanent of a matrix:In particular, this permanent lies between zero and one.

For a two-person example,The distinct outcomes $(1,2)$ and $(2,1)$ have probabilities $3/5$ and $1/20$. ThereforeThe colliding outcomes $(1,1)$ and $(2,2)$ have probabilities $3/20$ and $1/5$, totaling $0.35$. All four outcomes are shown below; their probabilities sum to one.

If every choice has the uniform distribution on a finite set of $n$ labels, then $p_{ij}=1/n$ and the distinct-choice probability is $n!/n^n$. For three choices and three labels it is $6/27=2/9$. This counts a subset of all $n^n$ ordered outcomes, not independent pairwise collision events. Independence of the original choices is essential: without it, the individual rows of $P$ do not determine the joint probabilities.

The sum of independent Bernoulli random variables with possibly unequal success probabilities $p_i$ has a Poisson binomial distribution. Its probability generating function is $G_K(z)={\prod }_i(1-p_i+p_{i}z)$: expand the product to read off the probability mass function. Linearity of expectation and additivity of variance for independent random variables give $\mathbb{E}K={\sum }_i{p}_i$ and $\mathrm{Var}(K)={\sum }_i{p}_i(1-p_i)$. Equal $p_i=p$ recover the binomial distribution.

For three trials with success probabilities $1/10,3/10,4/5$, expanding $(9/10+z/10)(7/10+3z/10)(1/5+4z/5)$ gives masses $0.126,0.572,0.278,0.024$ at $0,1,2,3$. Their expected value is $1.2$ and variance is $0.46$. Replacing all three success probabilities by their average $0.4$ preserves the expected value but gives binomial distribution variance $0.72$, so it does not preserve the law.

Let $N$ be a nonnegative integer-valued random variable with finite expected value $\mu$, and independently label each of its items with either of two equally likely labels. If the two resulting counts are independent random variables, then $N$ has a Poisson distribution. Indeed, its probability generating function obeys $G(s)=G((1+s)/2{)}^2$. With $H(t)=G(1-t)$, iteration gives $H(t)=H(t/2^k{)}^{2^k}$. Finite expected value gives $H(h)=1-\mu h+o(h)$, hence $H(t)=e^{-\mu t}$ and $G(s)=e^{\mu (s-1)}$. Conversely, Poisson thinning produces independent Poisson distributions for the two counts.

A Homogeneous Poisson process models arrivals at a constant rate $\lambda >0$. Its counting process $N_t=N(t)$ counts arrivals in $(0,t]$, with $N_0=0$. Its paths are right-continuous nondecreasing functions, take nonnegative integer values, and have unit jumps almost surely. The defining Poisson distribution of each increment isIt has stationary increments: the probability distribution above depends on the duration $t$, not the starting time $s$. It also has independent increments: counts in any finite collection of disjoint time intervals are independent random variables. Together these conditions specify the joint count laws, not merely the Poisson distribution at a single time.^[ref]

The rate has units of arrivals per unit time; $\lambda t$ is dimensionless. The Poisson distribution givesFor a toy rate $\lambda =2$ per minute and a duration of $1.5$ minutes, the expected value and variance of the count are both $3$. The probability of no arrival is $e^{-3}\approx 0.049787$, and the probability of exactly three is $3^3{e}^{-3}/3!=\frac{9}{2}{e}^{-3}\approx 0.224042$. In two consecutive one-minute intervals, the probability of exactly two arrivals in the first and exactly one in the second is $(2e^{-2})(2e^{-2})=4e^{-4}\approx 0.073263$, using independent increments. These are exact model calculations with rounded decimals, not measured frequencies.

The diagram's nine hand-chosen arrivals illustrate a path; the line $\lambda t$ is an expected value, not a prescribed trajectory. Constant rate does not imply evenly spaced arrivals. Poisson interarrival times connect the count description to waiting, while Poisson thinning explains how independently routing arrivals produces separate streams. This makes the Poisson process useful as a simple arrival model for a queue, provided constant rate and independent increments are plausible. For a deterministic changing rate, use an Inhomogeneous Poisson process; bunching or history-dependent arrivals need additional assumptions. Conditional on $N_t=n$, the arrival times are the order statistics of $n$ independent uniform distributions on $[0,t]$, as developed in Poisson process conditional arrival times.

Let $S,T$ be independent random variables with exponential distributions of respective positive rates $\alpha ,\beta$. Condition on $T=t$ and use the survival function of $S$:At $u=0$ this is $P(S>T)$. Dividing by that probability proves that the positive difference conditional on $S>T$ again has an exponential distribution with rate $\alpha$. The memorylessness of the exponential distribution persists after an independent competing event has already occurred. Replacing the difference by a negative sentinel when $S\le T$ adds an atom of a measure of mass $\alpha /(\alpha +\beta )$ at that sentinel without changing the positive tail.

For nonnegative integer-valued independent random variables $X,Y$, factor the expected value of a product:The Cauchy product explains what this means for individual probability masses:Every way of splitting the total contributes once. This is convolution of independent random variables; multiplying probability generating functions performs the bookkeeping automatically.

For an original example, take independent $X\sim \mathrm{Bernoulli}(0.3)$ and $Y\sim \mathrm{Binomial}(2,0.5)$, using the Bernoulli distribution and binomial distribution. ThensoDirect enumeration of the six pairs $(x,y)$ gives the same probability masses. For example, total one collects $(0,1)$ and $(1,0)$, with contributions $0.7\cdot 0.5+0.3\cdot 0.25=0.425$. The coefficients sum to one. The expected value is $G_{X+Y}^{\prime }(1)=1.3$; since $G_{X+Y}^{\prime \prime }(1)=1.1$, the variance is $1.1+1.3-1.3^2=0.71$, agreeing with $0.3\cdot 0.7+2\cdot 0.5\cdot 0.5$.

Uncorrelated random variables are insufficient for the product identity. For a concrete counterexample, let $U$ be uniform on $\{-1,0,1\}$ and set $A=U+1$, $B=U^2$. Then $\mathrm{Cov}(A,B)=\mathbb{E}[U^3]=0$, but $\mathbb{P}(A+B=0)=0$, whereas $G_A(0)G_B(0)=1/9$. Zero covariance controls one mixed moment, not the full joint probability mass function.

Probability theory describes uncertainty through a probability space: possible outcomes form a sample space, observable questions are events, and a probability measure assigns them numbers between zero and one. The Kolmogorov axioms make those assignments consistent. They let us combine information about several events, update it through conditional probability, and study random variables without listing every possible outcome separately.

For example, roll two dice whose results are independent random variables, each with a discrete uniform distribution on $\{1,\dots ,6\}$. There are 36 equally likely ordered outcomes. Only $(1,6),(2,5),\dots ,(6,1)$ have sum seven, so that event has probability $6/36=1/6$. The sum itself is not uniformly distributed: sum two has only one outcome. Choosing a probability distribution for the elementary outcomes is part of the model; equal likelihood does not follow merely from counting the named possibilities.

The subject becomes especially useful when complete outcome lists are enormous or infinite. Independence turns a joint calculation into simpler pieces, expected value summarizes a random variable, and the law of large numbers explains how averages can become predictable even when individual observations are uncertain. The central limit theorem describes the scale and shape of many remaining fluctuations. These are distinct claims: a small variance of an average does not make every observation close to its expected value.

There is a satisfying unity here: dice, hidden causes, and Brownian motion all use the same rules for events and conditional probability. A model still needs justification. Its exact deductions describe what its assumptions imply; comparison with observations belongs to statistical inference.

Suppose $X_N$ and $Y_N$ are independent random variables in Polish spaces, each satisfying a large deviation principle at speed $a_N$, with good rate functions $I$ and $K$. Their pair has good rate function $I(x)+K(y)$. On open rectangles, independence makes the logarithmic lower bounds add. For the upper bound, the marginal families are exponentially tight, and so is their pair. Cover a closed set intersected with the product of the two compact containment sets by finitely many small rectangles and add the corresponding local upper bounds. The finite union bound and then increasing the containment level give the closed-set bound. Goodness follows because $\{I+K\le M\}$ is closed inside $\{I\le M\}\times \{K\le M\}$.

For independent random variables $X,Y$ with the uniform distribution on $(0,b)$, use the probability density of a product of random variables and integrate $1/(b^{2}x)$ over $z/b<x<b$. The displayed probability density function is zero off $(0,b^2)$ and has integral one. More generally, for independent and identically distributed factors $U_j$ with this uniform distribution, $W_n={\prod }_{j=1}^n{U}_j$ satisfiesThe expectation of a product of independent random variables proves these identities directly.

Let $P$ be a nonzero multivariate polynomial over a field $K$, with total degree at most $d\ge 0$. Choose a nonempty finite set $S\subseteq K$. For independent random variables $a_1,\dots ,a_n$, each with a uniform distribution on a finite set on $S$, the Schwartz-Zippel lemma givesNonzeroness is essential; the zero polynomial always evaluates to zero. The total degree sums exponents within each monomial, rather than taking their maximum: $x^2{y}^3+y$ has total degree five. The field need not be finite, but $S$ must be finite. The estimate reduces error only when $d/|S|<1$.^[ref]

The proof begins with the root bound for a polynomial. For several variables, write $P={\sum }_{j=0}^m{Q}_j(x_1,\dots ,x_{n-1})x_n^j$, with $Q_m\ne 0$. The total degree of $Q_m$ is at most $d-m$. By induction, $Q_m$ vanishes with probability at most $(d-m)/|S|$. Otherwise the remaining univariate polynomial has at most $m$ polynomial roots. Adding the two contributions gives $d/|S|$.^[ref]

The algorithm samples a point, performs exact polynomial evaluation, rejects the identity if the value is nonzero, and otherwise accepts. The lemma bounds false acceptance for every fixed nonzero input. For the polynomial identity testing over a finite field example, $G$ has $d=2$ and $|S|=5$, so its exact $2/5$ error attains the bound. Numerical near-zero values from floating-point arithmetic do not satisfy this exact algebraic contract. A polynomial-time algorithm claim additionally needs efficient sampling and evaluation, including the bit cost of representing and operating on field elements. Counting field operations alone does not establish that cost.

A real-valued random variable is a measurable function $X$ from a probability space $(\Omega ,\mathcal{F},\mathbb{P})$ to the real numbers with their Borel sigma-algebra. More general targets give random elements. For every Borel set $A$, measurability ensures that $\{\omega :X(\omega )\in A\}$ is an event, so the question "does the observed value lie in $A$?" has a defined probability.

The function $X$ is fixed; its input outcome is uncertain. This separates what we measure from how the uncertainty is modeled. A probability distribution is the resulting pushforward measure, ${\mathbb{P}}_X(A)=\mathbb{P}(X\in A)$. A cumulative distribution function, probability mass function, or probability density function describes that measure in an appropriate setting. The random variable and its probability distribution are different objects: the former also retains its relationship to other measurements on the same sample space.

For two independent uniform dice, let $U,V$ record their results and define $S=U+V$ and $M=\max (U,V)$. The 36 ordered pairs are equally likely, but the resulting probability mass functions differ:For the sum, count the diagonal of the outcome grid having total $s$. For the maximum, subtract the $(m-1{)}^2$ pairs with both results below $m$ from the $m^2$ pairs with both results at most $m$. These examples show how a deterministic function transforms a simple input probability distribution into a nonuniform output one.

The corresponding expected values are $\mathbb{E}S=7$ and $\mathbb{E}M=161/36$. The first follows immediately from linearity of expectation; the second follows from ${\sum }_{m=1}^{6}m(2m-1)/36$. Neither number must be an attainable result. An expected value describes an average under the probability measure, not a prediction that the next observation will equal it.

The distinction between a random variable and its probability distribution matters for dependence. $U$ and $V$ have the same discrete uniform distribution and are independent random variables. $U$ and a second copy of $U$ also have that same marginal probability distribution, but they always agree. Their joint probability distribution is concentrated on the diagonal; the marginals alone cannot say which relationship holds. This is why a model for several measurements must specify their joint law or an appropriate independence assumption.

Indicator random variables turn events into numbers: $1_A$ equals one on $A$ and zero elsewhere, and $\mathbb{E}{1}_A=\mathbb{P}(A)$. This connects event counting with expected value and variance, and lets tools for random variables answer questions about probabilities. More complicated functions of observations can describe totals, waiting times, or estimation errors using the same framework.

Imagine an instrument measuring the same fixed quantity twice on each unit. Across units the persistent signal $T$ is a latent variable; the two measurement errors ${\epsilon }_1,{\epsilon }_2$ are fresh, mutually independent random variables, also independent of $T$. Assume they have mean zero and common finite variance $v$, while $T$ has mean $\mu$ and variance $s$. ThenThus the Pearson correlation coefficient is $\rho =s/(s+v)$ when $s+v>0$. Both measurements retain the same marginal mean and variance. Repetition does not make the entire population collapse toward $\mu$; it changes what we expect after selecting on $X$.

For $s,v>0$, further assuming normal distributions for $T$ and both measurement errors gives the conditional expectationSince $0<\rho <1$, this predicted reading is closer to $\mu$ than $x$ is. Independence and zero mean errors alone do not justify this linear conditional expectation: without the normal distribution assumption the same expression is the linear least-squares projection, and the actual conditional expectation can be nonlinear. Correlated errors, a changing signal, or different error variances require their own model; a noisy reading by itself supplies no universal shrinkage factor.

Replication in experimental design repeats a treatment on multiple independent experimental units, so that variation between those units can inform a treatment contrast and its standard error. Repeated measurements on one experimental unit can improve measurement precision without providing additional independent treatment replication. The experimental unit is identified by where the treatment is applied, rather than by how many responses are recorded.^[ref]

For a constructed example, grow one plant in each of eight separate pots. Apply treatment A to four pots and treatment B to four pots, without sharing drainage water between pots. Measure the length of three leaves per pot. The pot is the experimental unit; the leaf is the observational unit. There are four treatment replicates per group, eight experimental units overall, and $8\cdot 3=24$ measurements. The three leaves in one pot are nested measurements, because they share the same plant and treatment application. The setup assumes that pots do not affect one another; simply putting them in separate containers does not establish statistical independence.

Randomization chooses which experimental units receive which treatments; replication supplies multiple experimental units receiving the same treatment. These solve different problems. In the balanced allocation above, choose the four A pots uniformly among the $\left(\genfrac{}{}{0ex}{}{8}{4}\right)=70$ possible subsets. A particular pot has probability $1/2$ of receiving A. Given that one pot receives A, a second receives A with conditional probability $3/7$, rather than $1/2$. Thus the assignments are not independent random variables, even though the experimental units can supply independent responses conditional on those assignments in a suitable statistical model.

An actual ecology experiment illustrates why the distinction matters. In Stuart H. Hurlbert's account, twelve experimental ponds initially received zero, fifty or 450 fish, with five, four and three ponds respectively. Preparation included assessing variation between ponds before introducing fish; allocation considered both location and existing differences between ponds. The zero-fish ponds supplied the scientific control. These were repeated treatment applications to ponds, not twelve samples from one pond.^[ref]

The original article followed the southern California ponds for ten months. Its readouts included zooplankton populations and total phytoplankton; it reported decreases in some zooplankton populations and increased total phytoplankton.^[ref] Repeated sampling within a pond helps describe its response through time, while ponds supply treatment replication. Unequal group sizes and initial pond variation matter for the statistical model; the treatment counts alone do not establish equal variances or independent responses.

For independent random variables with a common law, $L_n={\min }_{j\le n}{X}_j$ satisfies $L_{n+1}=\min (L_n,X_{n+1})$. The next-state law therefore depends on the past only through $L_n$.

Draw two different labels uniformly from $\{1,2,3,4\}$ using sampling without replacement. Each position has a uniform distribution on a finite set, but $\mathbb{P}(X_1=X_2)=0$. Two independent random variables with those same marginal distributions would match with probability $1/4$. Equal marginal laws therefore do not make this sample an IID sample. Mark some of those four labels as successes. Their count in the two draws has a hypergeometric distribution; independent Bernoulli trials with a common success probability give a binomial distribution.

If $L_j=d{P}_j/dQ$ and the nonnegative mixture weights sum to one, the likelihood ratio of the mixture model is $L={\sum }_j{w}_j{L}_j$. Expanding ${\mathbb{E}}_Q{L}^2-1$ proves the displayed chi-squared divergence identity. For independent random variables sampled from a fixed component, one-observation cross moments multiply across observations. The component is drawn once for the whole sample; redrawing it per observation gives a different mixture model.

Choose one random variable $U$ with the continuous uniform distribution on $[0,1]$, and reuse it in every event:Each probability is $1/(n+1)$, so their harmonic series diverges, just as in summable and harmonic event probabilities. But the events are decreasing: $A_{n+1}\subseteq A_n$. For every outcome $U=u>0$, all indices with $n+1>1/u$ fail. Only finitely many events occur on that outcome. The limsup of events is exactly $\{U=0\}$, a zero-probability event. This is a concrete closed-interval version of the shrinking-interval dependence example.^[ref]

The missing assumption is visible already in two events. For $i<j$, their intersection is $A_j$, henceMoreover, the whole tail union is just $A_m$, with probability $1/(m+1)\to 0$. This calculation explains directly why divergence alone fails: overlapping events keep spending their individual probabilities on the same outcomes.

Let $N={\sum }_{n\ge 1}{1}_{A_n}$ be the total number of occurrences, a sum of indicator random variables. Although $N<\infty$ almost surely, Tonelli theorem gives $\mathbb{E}[N]={\sum }_{n\ge 1}1/(n+1)=\infty$. More explicitly, $\mathbb{P}(N\ge k)=1/(k+1)$ for $k\ge 1$: very small positive values of $U$ give very long, but finite, initial runs. Infinite expected value does not imply a positive probability of an infinite count.