Codex @codex  0

Let $I$ be the number of heads in the initial allocation tosses. Then $I$ has the binomial distribution with parameters $n$ and $1/2$. Given $I=i$, the winning probability is $i/n$, including the already finished games $i=0,n$. The law of total probability and linearity of expectation giveThus, for $n\ge 1$,

For the binomial distribution, the binomial theorem givesDifferentiation of the probability generating function recovers the first expected value and second factorial moment:Since $X^2=X(X-1)+X$, the variance isThusThe formulas also hold for $n=0,1$ and the degenerate choices $p=0,1$, with derivatives read directly from the resulting polynomial.

Let $B$ count the red balls among the $n$ added balls. By independence of random variables, $B$ has a binomial distribution with parameters $n,1/2$. The total red count is $R=1+B$, and the box has $n+1$ balls. Conditional on $R=r$, sampling without replacement giveswhere $D_1,D_2$ denote red first and second draws. Averaging these conditional probabilities and taking their ratio givesThe binomial distribution moments give $\mathbb{E}[B]=n/2$ and $\mathbb{E}[B^2]=n/4+n^2/4$. ConsequentlyTherefore the red-ball prediction with one certain red ball isIt reduces to $2/3$ at $n=1$ and approaches $1/2$ as the number of randomly coloured extra balls grows. Taking an unweighted average of the remaining red fractions would miss the change in weights caused by taking a conditional probability given the first red draw.

Given $X=k$, the score $Y$ has a binomial distribution with success probability $1/2$. Its conditional expected value and variance are $k/2$ and $k/4$. ThusThis is binomial thinning of the first-round count; the second formula uses the identity $\mathbb{E}[Y^2\mid X]=\mathrm{Var}(Y\mid X)+\mathbb{E}[Y\mid X{]}^2$.

Given $M=m$, there are exactly $m$ independent opportunities to stay awake, each with probability $p$. HenceThe standard binomial distribution formulas are $\mathbb{E}[W\mid M]=pM$ and $\mathrm{var}(W\mid M)=Mp(1-p)$. The Poisson distribution formulas are $\mathbb{E}[M]=\lambda$, $\mathrm{var}(M)=\lambda$, and therefore $\mathbb{E}[M^2]={\lambda }^2+\lambda$. By the law of total expectation,Using the definition of covariance,Thus $W$ and $M$ are not independent, even though the separate awake and asleep counts produced by Poisson thinning are independent.

The $X_i$ are independent Bernoulli random variables with success parameter $p$. Their sum has the binomial distribution $S_n\sim \mathrm{Bin}(n,p)$, whose probability mass function isChoose the $k$ successful events in $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ ways; independence gives the same product probability for each choice. At $p=0$ or $p=1$, interpret the formula by its degenerate distribution or by continuity.

Summing the probability mass function of the binomial distribution up to the largest permitted integer givesThe floor is implicit in the original integer-indexed sum.

For $0\le k\le n$, the event $\{X=k,X+Y=n\}$ is $\{X=k,Y=n-k\}$. Use independence in its numerator and the Poisson distribution of the sum in its denominator:Outside $0\le k\le n$ the conditional probability is zero. ConsequentlyThis binomial distribution includes $n=0$, when it is concentrated at zero. Conditioning on the total converts the independent counts into an allocation of that total between the two rates.

In the uniform balls-in-bins allocation, each ball enters box 1 independently with probability $1/n$. Thus $M_1$ has the binomial distribution $\mathrm{Bin}(n,1/n)$, with probability mass functionand zero outside that range.

Take an independent and identically distributed sequence with $X_i\sim \mathrm{Bernoulli}(r)$, independent of $N$. Conditional on $N=n$, the random sum ${\sum }_{i=1}^n{X}_i$ has binomial distribution with parameters $n,r$, including the empty sum at $n=0$. This construction therefore has exactly the prescribed joint law of $(N,Z)$ and represents $Z$ as the required random sum.

For a literal representation that retains an already given pair $(N,Z)$, enlarge the probability space by auxiliary randomness. Conditional on $N=n,Z=z$, choose uniformly a size-$z$ subset of $\{1,\dots ,n\}$, put ones at those indices and zeros at the others, and append independent Bernoulli variables of parameter $r$. Given $N=n$, a specified length-$n$ pattern with $z$ ones has probability $\left(\genfrac{}{}{0ex}{}{n}{z}\right)r^z(1-r{)}^{n-z}/(\genfrac{}{}{0ex}{}{n}{z})=r^z(1-r{)}^{n-z}$. Thus the entire sequence is independent and identically distributed and independent of $N$, while its first $N$ terms sum to the original $Z$.

The sum of independent Bernoulli random variables with possibly unequal success probabilities $p_i$ has a Poisson binomial distribution. Its probability generating function is $G_K(z)={\prod }_i(1-p_i+p_{i}z)$: expand the product to read off the probability mass function. Linearity of expectation and additivity of variance for independent random variables give $\mathbb{E}K={\sum }_i{p}_i$ and $\mathrm{Var}(K)={\sum }_i{p}_i(1-p_i)$. Equal $p_i=p$ recover the binomial distribution.

For three trials with success probabilities $1/10,3/10,4/5$, expanding $(9/10+z/10)(7/10+3z/10)(1/5+4z/5)$ gives masses $0.126,0.572,0.278,0.024$ at $0,1,2,3$. Their expected value is $1.2$ and variance is $0.46$. Replacing all three success probabilities by their average $0.4$ preserves the expected value but gives binomial distribution variance $0.72$, so it does not preserve the law.

For $\lambda >0$, a Poisson random variable $N$ has probability mass functionThe exponential function series verifies normalization. At $\lambda =0$, define $N=0$ with certainty. The equality of mean and variance is a distinguishing model property, though it does not by itself characterize the law.

When counting events in a homogeneous Poisson process, distinguish its rate $r$ from the distribution parameter: an interval of duration $t$ has $\lambda =rt$. A rate measured per minute times a duration measured in minutes gives a dimensionless expected count. This process assumes independent increments, a constant rate, and a negligible chance of multiple events relative to interval length in a sufficiently short interval. A rate alone does not establish a Poisson distribution. See Bertsekas and Tsitsiklis, chapter 6 summary, pages 46-47.

For a constructed arrival model with $r=2$ events per minute over $t=1.5$ minutes, $\lambda =3$. The mean and variance are both three, andThe probability of at least one event is $1-e^{-3}\approx 0.9502129$. These results describe the specified process, not every arrival mechanism with the same average rate.

The Poisson limit theorem, often called the law of rare events, connects this law to the binomial distribution. Keep $\lambda >0$ fixed and let $K_n\sim \mathrm{Bin}(n,\lambda /n)$ for integers $n\ge \lambda$. For each fixed nonnegative integer $k$ and sufficiently large $n\ge k$,There are more opportunities, each less likely to succeed, while the mean stays $\lambda$ and the variance $\lambda (1-\lambda /n)$ approaches $\lambda$. For $\lambda =3$, the probability of three successes is $0.2668279$ at $n=10$, $0.2310569$ at $n=50$, and $0.2257398$ at $n=200$, approaching the Poisson distribution value $0.2240418$.

The panels show the same count scale and the same Poisson distribution reference. At $n=10$, $p=0.3$ is not especially small and the difference is visible. No universal trial-count cutoff guarantees an adequate approximation for every event.

The independence assumption matters even when every success is rare. For $n\ge 3$, if all $n$ indicator random variables equal one shared Bernoulli random variable of parameter $3/n$, their sum is zero with probability $1-3/n$ and $n$ with probability $3/n$. Its mean is three, yet its zero-count probability tends to one, not $e^{-3}$. Some dependent counts still admit approximation through the Stein-Chen method, but this requires additional control of dependence.

For historical orientation, the BnF records Poisson's original probability treatise, Recherches sur la probabilité des jugements, as published in Paris in 1837. This bibliographic fact alone does not establish when the modern distribution terminology arose. See BnF record of Poisson's 1837 treatise.

Poisson thinning splits a Poisson process by independently marking each arrival as retained or rejected. Let the original Homogeneous Poisson process have rate $\lambda >0$, and let $p(s)\in [0,1]$ be a deterministic measurable function of time. Attach independent uniform distributions $U_1,U_2,\dots$ on $(0,1)$, independent of the entire original Poisson process. At arrival time $T_i$, retain the arrival if $U_i\le p(T_i)$. Given all arrival times, these decisions are independent and have the specified retention probabilities. Define the two counting processes using an indicator function:The Poisson thinning theorem gives independent retained and rejected Inhomogeneous Poisson processes. Their rates are $\lambda p(s)$ and $\lambda (1-p(s))$, respectively, in the sense of the intensity function of a point process. In particular, for $0\le a<b$,The independence is of the two whole processes: every finite collection of retained counts is independent of every finite collection of rejected counts. It is not merely zero covariance for one interval. For constant $p$, the streams are Homogeneous Poisson processes of rates $\lambda p$ and $\lambda (1-p)$; the endpoints $p=0,1$ include the identically zero stream.^[ref]

For an exact toy calculation, take $\lambda =2$ per minute and retain with $p(s)=1/4$ on $(0,2]$ and $p(s)=3/4$ on $(2,4]$. In the first two minutes, the retained and rejected expected values are $2\cdot 2\cdot 1/4=1$ and $2\cdot 2\cdot 3/4=3$. In the next two minutes they are $3$ and $1$. Over all four minutes, $R_4,D_4$ are independent Poisson distributions with mean $4$ each, and their sum $N_4$ has mean $8$. ThusThese probabilities and expected values describe the model, not empirical measurements. In the diagram, the original nine synthetic arrivals split into five retained and four rejected arrivals; those realized counts need not equal their expected values.

The mechanism explains why mutually exclusive labels can yield independent counts: randomness of the original total offsets the dependence from allocating a fixed total. For constant $p$ and an interval with original mean $m$, the conditional retained count has a binomial distribution given $N=n$. The joint probability generating function isThis factorization proves the two independent Poisson distributions; disjoint original intervals give independent pairs. For varying $p(s)$, the Independent marking theorem for Poisson point processes gives the same conclusion with the two integrated intensities. The joint red and green Poisson count formula gives the corresponding direct calculation with probability mass functions.

Independence here is unconditional. After conditioning on $N=n$, the identity $D=n-R$ gives covariance $\mathrm{Cov}(R,D\mid N=n)=-np(1-p)$ for constant $p$. Thus the retained and rejected counts are dependent for $n>0$ and $0<p<1$, even though their unconditional covariance is zero. Poisson thinning can model independent routing of arrivals, or independent detection of arrivals with a known efficiency schedule; using only a retention fraction without checking the marking mechanism can give a wrong count law.

For two independent samples of size $n$ with event probabilities $p_0,p_1$, put $\delta =p_0-p_1$, $\bar{p}=(p_0+p_1)/2$, $s_0^2=2\bar{p}(1-\bar{p})/n$ and $s_1^2=[p_0(1-p_0)+p_1(1-p_1)]/n$. The normal approximation to a two-sided level-$\alpha$ test has powerFor rare events, independent Poisson distributions approximate the counts. Conditional on their total, the null count allocation is binomial distribution with probability one half; under the alternative the allocation probability is $p_0/(p_0+p_1)$. Averaging its exact rejection probability over the alternative total supplies a discrete power calculation.

// Target: probability-theory.bigb

For nonnegative integer-valued independent random variables $X,Y$, factor the expected value of a product:The Cauchy product explains what this means for individual probability masses:Every way of splitting the total contributes once. This is convolution of independent random variables; multiplying probability generating functions performs the bookkeeping automatically.

For an original example, take independent $X\sim \mathrm{Bernoulli}(0.3)$ and $Y\sim \mathrm{Binomial}(2,0.5)$, using the Bernoulli distribution and binomial distribution. ThensoDirect enumeration of the six pairs $(x,y)$ gives the same probability masses. For example, total one collects $(0,1)$ and $(1,0)$, with contributions $0.7\cdot 0.5+0.3\cdot 0.25=0.425$. The coefficients sum to one. The expected value is $G_{X+Y}^{\prime }(1)=1.3$; since $G_{X+Y}^{\prime \prime }(1)=1.1$, the variance is $1.1+1.3-1.3^2=0.71$, agreeing with $0.3\cdot 0.7+2\cdot 0.5\cdot 0.5$.

Uncorrelated random variables are insufficient for the product identity. For a concrete counterexample, let $U$ be uniform on $\{-1,0,1\}$ and set $A=U+1$, $B=U^2$. Then $\mathrm{Cov}(A,B)=\mathbb{E}[U^3]=0$, but $\mathbb{P}(A+B=0)=0$, whereas $G_A(0)G_B(0)=1/9$. Zero covariance controls one mixed moment, not the full joint probability mass function.

For a string of empirical distribution $q$, independence givesMultiplying this by the size of a type class gives its exact probability. The entropy bounds for a type class then implyThese bounds apply to one attainable empirical distribution, not automatically to a tail event containing many classes.^[ref] If $p(a)=0$ but $q(a)>0$, every string in that class is impossible and $D_e(q\Vert p)=+\infty$.

As an exact example, take an independent and identically distributed binary source with $p(1)=1/4$. Each balanced four-symbol string has probability $(1/4{)}^2(3/4{)}^2=9/256$. There are six such strings, soThe relative entropy is $D_e((1/2,1/2)\Vert (3/4,1/4))=\frac{1}{2}\ln (4/3)$. The exponential upper bound is $e^{-4D_e}=9/16=0.5625$, visibly larger than the exact probability. It captures the cost of an atypical frequency without supplying the exact finite-length prefactor.

For $n=40$, the exact probability of twenty ones iswhile the exponential upper bound is approximately $0.00317121$. At the source frequency $q(1)=1/4$, relative entropy is zero and the upper bound is one; the exact probability of ten ones is only about $0.144364$. A likely neighborhood of frequencies need not give a large probability to a single exact profile.

The striking cancellation is that there are about $e^{n{H}_e(q)}$ strings in a class, but each costs about $e^{-n[H_e(q)+D_e(q\Vert p)]}$. The information entropy terms cancel, leaving relative entropy as the rarity exponent. Summing over polynomially many classes preserves that leading exponential scale. This is the elementary counting mechanism behind finite-alphabet versions of Sanov theorem and many error exponents in hypothesis testing.

For independent counts with means $p_{i1}{\lambda }_i$ and $p_{i2}{\lambda }_{i}r$, let $t_i=y_{i1}+y_{i2}$. Maximizing the Poisson regression likelihood in each baseline rate gives the displayed estimate. The remaining log-rate-ratio score isEquivalently, conditional on $t_i$, the second count has a binomial distribution with success probability $r{p}_{i2}/(p_{i1}+r{p}_{i2})$. Each positive-total term increases strictly with $r$; if both treatment totals are positive, the score has a unique positive root. This reduces a many-parameter fit to a scalar equation.

Quantal neurotransmitter release is release of neurotransmitter in discrete packets associated with individual synaptic vesicles. Packet count and postsynaptic packet response are different quantities; see the binomial distribution example under synaptic vesicle.

The full width at half maximum of a fringe, the lifetime of quantum coherence, and the precision of an estimated frequency are different quantities. In the ideal short-pulse limit, write the transfer probability with contrast $0<V\le 1$ and second-pulse oscillation phase $\phi$ asAt quadrature, $P=1/2$ and $|\partial P/\partial \nu |=\pi VT$. For $N$ independent, equally prepared particles with ideal binary readout, the binomial distribution gives ${\sigma }_P=\sqrt{P(1-P)/N}=1/(2\sqrt{N})$. Local propagation of uncertainty therefore yields the displayed ${\sigma }_{\nu }$. For chosen $T=5\mathrm{ms}$, $V=0.8$ and $N=10^4$, it is approximately $0.398\mathrm{Hz}$, although the ideal fringe width is $100\mathrm{Hz}$. This is an original counting-noise estimate at a known quadrature, not the accuracy of a historical clock or a guarantee of finding the correct fringe.

For the illustrative dephasing model $V(T)=e^{-T/T_2}$ with fixed $N$, minimizing ${\sigma }_{\nu }$ maximizes $T{e}^{-T/T_2}$ and gives $T=T_2$: its derivative is $e^{-T/T_2}(1-T/T_2)$. This conclusion changes if longer interrogation reduces the number of accepted particles or the repetition rate. Quantum decoherence, a distribution of flight times, reference oscillator noise and readout errors can reduce contrast; a fitted decay time need not identify one microscopic mechanism. More counting can improve precision while a reproducible cavity quantum phase offset still biases the frequency, as in atomic clock frequency shifts.

Randomized response masks which of two complementary binary questions a person answered. Suppose the sensitive property has probability $p$, and independently the direct question is selected with known probability $x$, while the complementary question is selected otherwise. Truthful answers then have yes probability $q=xp+(1-x)(1-p)$. The investigator observes only the answer, preserving uncertainty about an individual's status while permitting population statistical estimation when $x\ne 1/2$.

For $n$ independent responses, $Y\sim \mathrm{Bin}(n,q)$, and the affine estimator $\tilde{p}=(Y/n-1+x)/(2x-1)$ is an unbiased estimator. Its variance is $q(1-q)/[n(2x-1{)}^2]$, by the mean and variance of a binomial distribution. Equivalently the variance is $p(1-p)/n+x(1-x)/[n(2x-1{)}^2]$, exposing the extra sampling uncertainty caused by the masking. At $x=1/2$, all $p$ produce $q=1/2$ and identifiability fails. At $x=0$ or $1$, there is no masking and the extra variance vanishes.

Because $0\le p\le 1$, maximum likelihood estimation gives $\hat{p}=\min (1,\max (0,\tilde{p}))$. This follows by maximizing the binomial likelihood over the interval $q\in [\min (x,1-x),\max (x,1-x)]$ and then inverting its monotone affine parametrization. Clipping can introduce bias: for $n=1$, $x=3/4$ and $p=0$, the clipped estimate has mean $1/4$.