Discrete probability distribution

A discrete probability distribution is supported on a finite or countable set.

Benford law assigns leading base-ten digit $d\in \{1,\dots ,9\}$ probability ${\log }_{10}(1+1/d)$. This distribution arises when the fractional part of a base-ten logarithm is uniform: digit $d$ corresponds to the interval $[{\log }_{10}d,{\log }_{10}(d+1))$.

A decimal significand lies in $[1,10)$, obtained by removing the integer power of ten from a positive number. Multiplication by a constant rotates the fractional logarithm modulo one. Invariance under every such rotation makes the fractional logarithm uniform and hence implies Benford law. This is an invariant significand distribution, not a proper scale-invariant distribution for the entire positive number.

The leading decimal digit is determined by the fractional part of ${\log }_{10}X$. A uniform fractional logarithm gives Benford law by the lengths of the intervals $[{\log }_{10}i,{\log }_{10}(i+1))$. A log-uniform distribution whose logarithmic span is an integer has this property. Arbitrary partial-decade truncations generally do not.

If $a\ge 2$ is an integer with irrational ${\log }_{10}a$, then the leading digit $d$ of $a^n$ has limiting frequency ${\log }_{10}(1+1/d)$. Indeed the logarithmic fractional parts are the orbit of zero under an irrational rotation of the circle, so everywhere interval frequency under an irrational rotation applies. Thus for $a=2$, digit seven has frequency $(\log 8-\log 7)/\log 10$.

The discrete uniform distribution on a finite set $S$ assigns probability $1/|S|$ to every element of $S$.

An ordered sample from $n$ equally likely values is nondecreasing exactly when it is the unique sorted representative of its multiplicities. Stars and bars counts the nonnegative multiplicities summing to $r$; repeated values are included automatically.

A Bernoulli random variable equals one with probability $p$ and zero with probability $1-p$. Its mean is $p$ and its variance is $p(1-p)$.

If an event occurs with probability $p$ and is independently retained with conditional probability $q$, its retained indicator is Bernoulli with probability $pq$. Applying this independently to a binomial count turns $\mathrm{Binomial}(n,p)$ into $\mathrm{Binomial}(n,pq)$.

A geometric distribution models the number of independent Bernoulli trials needed to obtain the first success. On the convention supported by $1,2,\dots$, its mass is $(1-p{)}^{k-1}p$ and its mean is $1/p$. On the failures-before-first-success convention supported by $0,1,\dots$, its mass is $(1-p{)}^{k}p$ and its mean is $(1-p)/p$. Both conventions have variance $(1-p)/p^2$.

For a positive-integer geometric distribution $P(N=j)=p{q}^{j-1}$ with $q=1-p$, the capped time has masses $p{q}^{j-1}$ for $1\le j<k$ and $q^{k-1}$ at $k$. The last mass includes all later arrivals, not only $N=k$. Its probability generating function and expected value areThe expectation follows either by differentiating the PGF or summing the tail probabilities up to $k$.

When independent uniform draws are made from $n$ types, the time $T_n$ until every type has appeared decomposes as a sum of independent geometric waiting times with parametersConsequentlyand Chebyshev's inequality gives $T_n/(n\log n)\to 1$ in probability.

For uniform independent draws of $n$ coupon types, the waiting times with $r$ unseen types remaining are independent positive-support geometric distributions of parameter $r/n$. Summing their means gives $n{\sum }_{r=1}^{n}1/r$, and summing their variances gives the displayed formula. Independence follows from fresh draws after each discovery and the fact that the next stage's conditional distribution depends only on the count of remaining types.

On the failures-before-the-$r$th-success convention, a negative binomial variable with success probability $p$ has probability mass functionIt has expected value $r(1-p)/p$ and variance $r(1-p)/p^2$.

For the $r$th success to occur after exactly $x$ failures, the final trial must be a success, while the preceding $x+r-1$ trials contain $x$ failures and $r-1$ successes. Choosing the failure locations gives the factor $\left(\genfrac{}{}{0ex}{}{x+r-1}{x}\right)$.

If $Y\mid \Lambda \sim \mathrm{Poisson}(\Lambda )$ and $\Lambda$ has a gamma distribution, then $Y$ has a negative binomial distribution. For gamma shape $\nu$ and scale $\alpha$,

The binomial distribution counts successes in independent Bernoulli trials with common success probability.

For $W\sim \mathrm{Bin}(n,p)$, expand the fourth power of the sum of centered independent Bernoulli variables. Terms with an index occurring once vanish; single-index fourth moments contribute $nq(1-3q)$ and paired second moments contribute $3n(n-1)q^2$. Their sum is the displayed formula. Scaling by $n^4$ yields a uniform $O(n^{-2})$ fourth moment for $W/n-p$.

For independent Bernoulli observations with success probability $p$, the sample proportion is the success count divided by the sample size. It is an unbiased estimator with variance $p(1-p)/n$. It supplies the arm-specific risks in a risk ratio and the mean difference in a two-proportion comparison.

If $x$ successes occur in $n$ independent Bernoulli trials, the binomial likelihood for the success probability $p$ is proportional to $p^x(1-p{)}^{n-x}$.

If a sample of size $r$ is drawn uniformly without replacement from a population of size $n$ containing $c$ marked objects, the number $K$ of marked objects sampled has mass

The multinomial distribution gives the category counts from $n$ independent trials with category probabilities $p_1,\dots ,p_k$.

The vector of category counts is the sum of independent categorical indicator vectors. Their means are $p$ and covariance is $\mathrm{diag}p-p{p}^T$, so the multivariate central limit theorem gives the displayed limit. At uniform probabilities on $k$ categories, Pearson-standardized deviations have covariance equal to the orthogonal projection onto the $(k-1)$-dimensional subspace of zero-sum vectors. Their squared norm has an asymptotic chi-squared distribution with $k-1$ degrees of freedom.

Place $n$ labelled balls independently into $m$ labelled bins, each with probability $1/m$. The vector of bin counts has a multinomial distribution, and each count separately has the binomial distribution $\mathrm{Bin}(n,1/m)$. Distinct bin counts have covariance $-n/m^2$, so they are dependent for $n>0$ and $m>1$. Counts of bins of a specified size are sums of indicator variables; their expectations follow from the linearity of expectation, without any independence between bins.

In a uniform balls-in-bins allocation with $m\to \infty$ and $n/m\to \lambda \in (0,\infty )$, a fixed bin count has convergence in distribution to a Poisson distribution variable of mean $\lambda$ by the Poisson limit theorem. If $N_j$ counts bins containing exactly $j$ balls, symmetry and the linearity of expectation give $\mathbb{E}[N_j]/m=\mathbb{P}(B_1=j)\to e^{-\lambda }{\lambda }^j/j!$ for each fixed $j$. In particular the expected empty fraction tends to $e^{-\lambda }$, and the expected fraction with at least two balls tends to $1-(1+\lambda )e^{-\lambda }$. These are statements about expectations; convergence of the random fractions requires an additional argument.

For category counts $y_1,\dots ,y_k$ with fixed total $n$ and probabilities $p_1,\dots ,p_k$, the multinomial likelihood isIts maximum-likelihood fitted probabilities are ${\hat{p}}_j=y_j/n$.

For counts with a multinomial distribution, this is twice the log likelihood ratio of the saturated cell-probability estimate to fixed probabilities $p_i$. Zero-count terms have limiting value zero. Simulation under the specified probabilities gives a direct reference distribution when asymptotic approximations are unsuitable.

The categorical distribution describes one draw from finitely many categories with probabilities $p_1,\dots ,p_k$.

The Poisson distribution with mean lambda assigns mass $e^{-\lambda }{\lambda }^k/k!$.

A Poisson distribution of parameter $a$ conditioned to lie in $\{0,\dots ,C\}$. It is the occupancy law of a unit-resource loss network. Its mean is the carried load of an Erlang loss resource, $a[1-E(C,a)]$. Differentiating the mean with respect to $\log a$ gives its variance, which is positive for $a>0$ and $C\ge 1$; this proves strict monotonicity of the carried load.

A model with $Y\mid \Lambda \sim \mathrm{Poisson}(\Lambda )$ for a nonnegative random intensity. Conditional expectation and total variance give $\mathbb{E}Y=\mathbb{E}\Lambda$ and $\mathrm{Var}Y=\mathbb{E}\Lambda +\mathrm{Var}\Lambda$, explaining overdispersion. A Poisson-gamma mixture gives the negative binomial distribution.

For a uniform prior distribution on $(l,u)$ with $0<l<u$, conditionally independent Poisson distribution observations have sufficient count $s={\sum }_i{x}_i$. Their Bayesian posterior density is proportional to $t^s{e}^{-nt}$ on that bounded interval. The displayed posterior mean is the Bayes estimator under squared error loss, and is generally not the affine Bühlmann credibility estimate.

When the random intensity is $\alpha +Y$ with $Y$ having an exponential distribution of rate $\beta$, the mixed count is the sum of independent Poisson distribution and support-zero geometric distribution counts. The Poisson parameter is $\alpha$ and the geometric success probability is $\beta /(\beta +1)$. At $\alpha =0$ the law is geometric.

For the shifted-exponential Poisson mixture, the displayed relation holds for $n\ge 2$, initialized by $p_0=e^{-\alpha }\beta /(\beta +1)$ and $p_1=(\alpha +1/(\beta +1))p_0$. It follows by differentiating the probability generating function and comparing coefficients. Nonnegativity follows from the independent Poisson-geometric convolution of independent random variables representation.

A Poisson distribution conditioned on a positive count has probability mass function $\mathbb{P}(Y=y)={\theta }^y/[y!(e^{\theta }-1)]$ for $y\ge 1$ and $\theta >0$.

For the zero-truncated Poisson distribution, $21_{\{Y\text{ even}\}}$ is the unique integrable nonrandomized unbiased estimator of $p=1-e^{-\theta }$. Indeed unbiasedness gives ${\sum }_{y\ge 1}t(y){\theta }^y/y!=e^{\theta }+e^{-\theta }-2$, and power series coefficients force the parity values. Its variance is $2p-p^2$; clipping it to $1_{\{Y\text{ even}\}}$ gives mean squared error $p/2$, strictly smaller for $0<p<1$.

The sum of independent Poisson random variables is Poisson, with parameter equal to the sum of their parameters. This follows by multiplying their probability generating functions $\exp ({\lambda }_i(z-1))$.

For Bernoulli random variables $X_1,\dots ,X_n$, which need not be independent, put $p_i=\mathbb{P}(X_i=1)$, $S={\sum }_i{X}_i$, and $\lambda ={\sum }_i{p}_i$. ThenThe dependence penalty is the total correlation. The proof compares the joint Bernoulli law with the product of Poisson laws of means $p_i$, then applies the data processing inequality for relative entropy to addition.

A Poisson observation model represents a nonnegative count as conditionally Poisson with a mean supplied by an underlying process or regression model.

A Poisson observation is an integer-valued count modeled by a Poisson distribution of specified intensity, possibly conditionally on a latent field. Independent Poisson observations motivate Poisson data fidelity through their log-likelihood.

A zero-inflated Poisson distribution is a mixture of a point mass at zero and a Poisson distribution. If the Poisson component has weight $\pi$ and mean $\lambda$, then its mean is $\pi \lambda$ and its variance is $\pi \lambda +\pi (1-\pi ){\lambda }^2$.

A zero-inflated Poisson regression specifies a structural-zero probability $\pi$ and a susceptible-component Poisson distribution mean $\mu$. In this parametrization,The mean is $(1-\pi )\mu$. Logistic predictors for $\pi$ and logarithmic predictors for $\mu$ allow covariates to affect component membership and count intensity differently. A susceptible individual can still generate a zero count; susceptibility is not equivalent to an observed positive response.

In a zero-inflated Poisson regression, Bayes theorem separates structural zeros from susceptible zero counts. A positive count rules out the structural-zero class; a zero raises its posterior probability according to the displayed formula, but usually does not determine class membership with certainty. This same probability supplies the E-step of the EM algorithm for zero-inflated Poisson regression.

A logarithmic count-component contrast $b$ multiplies the susceptible mean by $e^b$, while a zero-logit contrast $g$ multiplies the structural-zero odds by $e^g$. If the baseline zero predictor is $\eta$, the marginal mean ratio isThus the count ratio alone is not a population-average effect when a covariate changes both components. With interaction terms, the relevant count contrast must first include the interactions for the stated reference group.

The expectation-maximization algorithm introduces $Z_i=1$ for a structural zero. Its E-step gives ${\tau }_i=0$ at positive counts and ${\tau }_i={\pi }_i/[{\pi }_i+(1-{\pi }_i)e^{-{\mu }_i}]$ at zero. The M-step maximizes a logistic log-likelihood with fractional responses ${\tau }_i$ and a Poisson log-likelihood with weights $1-{\tau }_i$. With only a binary predictor, write $n_j$ for group size, $S_j={\sum }_{x_i=j}{\tau }_i$ and $C_j={\sum }_{x_i=j}{Y}_i$; the explicit updates are ${\pi }_j=S_j/n_j$ and ${\mu }_j=C_j/(n_j-S_j)$. Log and logit group contrasts recover the regression coefficients, with boundary values interpreted through limits.

If $N_{\lambda }\sim \mathrm{Pois}(\lambda )$ and $\lambda \to \infty$, thenIndeed, its characteristic function is

If $X_{\lambda }$ has a Poisson distribution, then $\mathbb{P}(X_{\lambda }\le \lfloor \lambda \rfloor )\to 1/2$ as $\lambda \to \infty$. The Poisson central limit theorem standardizes by mean $\lambda$ and variance $\lambda$; the cutoff $(\lfloor \lambda \rfloor -\lambda )/\sqrt{\lambda }$ tends to zero. Squeezing between fixed nearby cutoffs and using continuity of the standard normal distribution function gives the limit.

Independent Poisson counts conditioned on their sum have a multinomial distribution.

For each group $g$, let independent cell counts have Poisson means ${\mu }_{gj}$. Conditional on their total $n_g$, the cells have a multinomial distribution with probabilitiesIncluding a free group main effect in the Poisson log-linear model makes its profiled likelihood for the remaining parameters proportional to the corresponding multinomial likelihood. The two fits therefore give the same fitted proportions and likelihood-ratio comparisons.

For positive group totals, the improper prior $du/u$ on an independent Poisson baseline yields the multinomial likelihood kernel after integration. This prior is flat in $\log u$. A proper finite-variance log-rate prior generally introduces an additional coefficient-dependent factor.

A mean-zero normal distribution prior of variance $s^2$ on the log baseline multiplies the multinomial kernel by the displayed correction, up to a constant independent of coefficients. The factor is not constant in $S$, so the posterior equivalence is approximate. Dominated convergence makes it tend to one as $s^2$ tends to infinity.

For large $\lambda$, a Poisson random variable of mean $\lambda$ is approximately normal with mean and variance $\lambda$.