Probability distribution

A probability distribution assigns probabilities to the possible values of a random variable.

A probability distribution is non-degenerate when it is not a point mass. This condition excludes constant limits in limit theorems and in the maximum domain of attraction of a distribution function.

For a probability measure on a topological space, its support consists of the points whose every open neighborhood has positive probability. For a continuous effort distribution supported on an interval, every open subinterval of that interval has positive probability.

An effort support is the support of a probability distribution used to randomize effort in a contest. The all-pay indifference equation with random entry determines this support together with its distribution function.

For finite fourth moment and positive variance, kurtosis is $\mathbb{E}[(X-\mathbb{E}X{)}^4]/\mathrm{Var}(X{)}^2$. The normal distribution has kurtosis $3$. A random scale can increase kurtosis even when the conditional distribution is normal.

For a positive random variable $X$ with finite nonzero expected value $\mu$, its integrated tail law has density $f_I(x)=\mathbb{P}(X>x)/\mu$ on $x\ge 0$. The tail integral formula for moments normalizes it. For $r>0$ its exponential transform is $\int e^{rx}{f}_I(x)dx=[M_X(r)-1]/(\mu r)$, with extended-value interpretation when needed.

A mixture distribution first chooses a component $j$ with nonnegative mixture weights $w_j$ summing to one, then samples from its law $P_j$. Consequently $P(A)={\sum }_j{w}_j{P}_j(A)$, and any finite moment-generating function is the same weighted sum of the component transforms. A component may be a Dirac measure, so a mixture may have both discrete and continuous parts.

A finite mixture distribution of exponential distributions with rates $r_i>0$ and nonnegative weights summing to one. Its expected value is ${\sum }_i{w}_i/r_i$ and its moment-generating function is ${\sum }_i{w}_i{r}_i/(r_i-t)$ on the common finite domain. The mixture chooses one component for each observation; it is not a sum of independent exponential variables.

A probability distribution is nonarithmetic when it is not concentrated on $d\mathbb{Z}$ for any $d>0$. Every law with a probability density is nonarithmetic. This hypothesis removes lattice oscillations from the key renewal theorem.

The right endpoint of a distribution with distribution function $F$ is $x_F=\sup \{x:F(x)<1\}$, possibly infinite. The survival function is positive below $x_F$.

The joint probability distribution of random variables $X_1,\dots ,X_n$ is the probability distribution of the random vector $(X_1,\dots ,X_n)$. It records both the marginal distributions and their dependence.

A family of random variables is exchangeable if its joint probability distribution is unchanged by finite permutations of its coordinates. Independent and identically distributed random variables are exchangeable, but exchangeable random variables need not be independent random variables.

If independent real random variables $X$ and $Y$ have densities $f_X$ and $f_Y$, their sum has the convolution density

A marginal distribution is the distribution of a selected coordinate or subvector of a random vector, obtained from the joint distribution by integrating or summing over the remaining coordinates.

Marginalization sums or integrates a joint probability distribution over unwanted coordinates to obtain a marginal distribution.

A marginal density is a probability density function for one component of a joint probability distribution. Integrating the joint probability density over the other coordinates gives it; for two components, $f_X(x)=\int f_{X,Y}(x,y)dy$.

A real probability distribution is symmetric about $m$ when $X-m$ and $m-X$ have the same distribution. A centered normal distribution is symmetric about zero.

A probability distribution on a vector space is centrally symmetric about zero when $X$ and $-X$ have the same distribution. Independent observations from such a distribution are jointly invariant under coordinatewise sign changes.

A Rademacher random variable takes the values $1$ and $-1$, each with probability $1/2$.

A Rademacher sum is a linear combination of independent Rademacher random variables, with scalar or normed-space-valued coefficients. Its moments are controlled by the Khintchine inequality in the scalar case and the Kahane-Khintchine inequality in a general normed vector space.

On the Boolean hypercube $\{-1,1{\}}^d$ with uniform measure, each coordinate is a Rademacher random variable. Products of these sign coordinate functions are Walsh characters. This use of Bernoulli function is distinct from the fluid-mechanical Bernoulli function.

For independent Rademacher random variables and $\Vert u_i{\Vert }_2\le 1$, this bound holds for $\alpha >0$. The moment-generating function satisfies ${\prod }_j\cosh (t{u}_{ij})\le e^{t^2/2}$, with strict inequality for a nonzero vector and $t>0$. Applying the exponential Markov bound to both tails and then the union bound proves a strict failure bound; a zero vector has failure probability zero. At $\alpha =\sqrt{2\log (2m)}$, the success probability is strictly positive, ensuring a sign vector with all linear forms below that threshold.

A truncated distribution is a conditional probability distribution obtained by restricting a random variable to a specified subset of its original support and renormalizing.

For a nonnegative integer claim count $N$ with $p=\mathbb{P}(N=0)<1$, the conditional law given $N>0$ has masses $\mathbb{P}(N=n)/(1-p)$ for $n\ge 1$. Its probability generating function is $(G_N(z)-p)/(1-p)$. Truncation at zero is meaningful only when the positive-count event has positive probability.

A truncated normal distribution is a normal distribution conditioned to lie in a specified subset of the real line. Its density is the original Gaussian density divided by the probability of the retained set.

To draw $N(\mu ,1)$ restricted to a positive value, let $a=\Phi (-\mu )$ and return $\mu +{\Phi }^{-1}(a+(1-a)U)$ for uniform $U\in (0,1)$. For restriction to a negative value use $\mu +{\Phi }^{-1}(aU)$. These are inverse transform sampling constructions. Repeatedly drawing an untruncated normal until its sign agrees is also exact, but inefficient for rare signs.

The beta distribution on $[0,1]$ has density proportional to $x^{\alpha -1}(1-x{)}^{\beta -1}$ and mean $\alpha /(\alpha +\beta )$.

The cumulative distribution function of a real random variable $X$ isIt is nondecreasing, right-continuous, tends to zero at $-\infty$, and tends to one at $+\infty$.

For $F_{\mu }(t)=\mu ((-\infty ,t])$, every difference quotient at $x$ is nonnegative and bounded by $4\mu (B(x,2|h|))/{\lambda }_1(B(x,2|h|))$. The spherical derivative of a singular measure vanishes, so the two-sided derivative is zero Lebesgue almost everywhere. A singular distribution function need not be constant: differentiation almost everywhere does not recover increments without absolute continuity of a function of the function.

The Kolmogorov distance between two probability distributions with cumulative distribution functions $F,G$ is ${\sup }_{z\in \mathbb{R}}|F(z)-G(z)|$.

If a cumulative distribution function is constant between consecutive lattice points spaced by $h$ near zero, then its Kolmogorov distance from the standard normal distribution function is at least half the increase of that function across one such interval. This gives a lower bound of order $h$.

If $X$ has a continuous cumulative distribution function $F$, then $F(X)$ has the uniform distribution on $(0,1)$. Combined with inverse transform sampling, this constructs the monotone rearrangement $G^{-1}(F(X))$ to a target with quantile function $G^{-1}$.

For correct continuous conditional forecasts, the conditional probability integral transform is uniform given the entire past. Iterating conditional probabilities makes the transformed sequence independent and uniform. Uniformity and serial checks assess absolute predictive calibration using only the forecast distributions and observations.

If distribution functions $F_n$ converge pointwise to a continuous distribution function $F$, then the convergence is uniform. Choose finitely many points whose successive $F$-increments and two tails are small, then use monotonicity to squeeze $F_n-F$ between its values at adjacent points.

For real observations $X_1,\dots ,X_n$, the empirical distribution function is

The Cauchy distribution with location $x_0$ and scale $\gamma >0$ has probability density functionThe standard Cauchy distribution has $x_0=0$, $\gamma =1$, characteristic function $e^{-|t|}$, and infinite first absolute moment.

The standard Cauchy distribution is $\mathrm{Cauchy}(0,1)$, with density $1/[\pi (1+x^2)]$ and characteristic function $e^{-|t|}$.

For independent Brownian motions $W,B$, the strong law for Brownian motion makes ${\int }_0^{\infty }{e}^{2W_s-s}ds$ finite almost surely, so bracket localization gives an almost sure limit of the stochastic integral. The arctangent transform of a two-noise affine diffusion and endpoint convergence of a bounded angle diffusion identify its distribution function as $1/2+\arctan (x)/\pi$. Conditional Gaussianity rules out atoms.

For $X\sim \mathrm{Cauchy}(x_0,\gamma )$,In particular, the Standard Cauchy distribution has characteristic function $e^{-|t|}$.

A Cauchy random variable is a random variable having a Cauchy distribution. For a standard Cauchy random variable $V$,

If $X_1,\dots ,X_n$ are independent standard Cauchy random variables, thenis again standard Cauchy. Indeed, its characteristic function is $(e^{-|t|/n}{)}^n=e^{-|t|}$.

A probability distribution is stable when every linear combination of independent copies has the same distribution up to a location and scale change. The Cauchy distribution is stable with stability parameter one.

A probability distribution is strictly stable with index $\alpha$ if $n^{-1/\alpha }(T^{(1)}+\cdots +T^{(n)})\stackrel{d}{=}T$ for every positive integer $n$, for independent copies. Here no centering term is allowed, unlike in the general definition of a stable distribution.

For a nonnegative integer-valued random variable $X$, its probability generating function isThe generating function of a sum of independent random variables is the product of their generating functions.

For a discrete random variable $X$, its probability mass function isIt is nonnegative and sums to one over the possible values of $X$.

For a distribution function $F$, its generalized inverse or quantile function is

For $Q(u)=\inf \{x:F(x)\ge u\}$, $0<u<1$, the quantile function is a monotone function and is left-continuous. The distribution function is right-continuous, giving $Q(u)\le x$ exactly when $u\le F(x)$. If ${\sup }_{v<u}Q(v)<Q(u)$, a point between them has $F(x)<u$ and contradicts this equivalence for some $v\in (F(x),u)$. A jump just to the right of $u$ is possible when $F$ is flat on an interval.

If $q$ is the quantile function of an absolutely continuous probability distribution with probability density function $f$, then $q^{\prime }(u)=1/f(q(u))$ for almost every $u\in (0,1)$. Gaps in the support may give jumps in $q$, so the identity concerns its ordinary almost-everywhere derivative. It follows by differentiating the inverse relation to the cumulative distribution function.

If $U$ is uniform on $[0,1]$, then $F^{-1}(U)$ has distribution function $F$. Indeed, right continuity givesapart from irrelevant endpoint cases.

For positive mixture weights $w_j$ summing to one, set $C_j={\sum }_{i\le j}{w}_i$ and choose $J=j$ when $C_{j-1}\le U<C_j$, where $U$ has a uniform distribution. Then $V=(U-C_{J-1})/w_J$ has a uniform distribution independently of $J$: for $0\le v\le 1$, $\mathrm{Pr}(J=j,V\le v)=w_{j}v$. This permits exact finite mixture model sampling with a recycled continuous random variable.

If $F_n\to F$ at every continuity point of $F$, then the generalized inverses satisfyat every continuity point of $F^{-1}$. Since a monotone function has only countably many discontinuities, evaluating all quantiles at one uniform random variable gives an almost-sure coupling of the corresponding convergence in distribution.

A discrete probability distribution is supported on a finite or countable set.

Benford law assigns leading base-ten digit $d\in \{1,\dots ,9\}$ probability ${\log }_{10}(1+1/d)$. This distribution arises when the fractional part of a base-ten logarithm is uniform: digit $d$ corresponds to the interval $[{\log }_{10}d,{\log }_{10}(d+1))$.

A decimal significand lies in $[1,10)$, obtained by removing the integer power of ten from a positive number. Multiplication by a constant rotates the fractional logarithm modulo one. Invariance under every such rotation makes the fractional logarithm uniform and hence implies Benford law. This is an invariant significand distribution, not a proper scale-invariant distribution for the entire positive number.

The leading decimal digit is determined by the fractional part of ${\log }_{10}X$. A uniform fractional logarithm gives Benford law by the lengths of the intervals $[{\log }_{10}i,{\log }_{10}(i+1))$. A log-uniform distribution whose logarithmic span is an integer has this property. Arbitrary partial-decade truncations generally do not.

If $a\ge 2$ is an integer with irrational ${\log }_{10}a$, then the leading digit $d$ of $a^n$ has limiting frequency ${\log }_{10}(1+1/d)$. Indeed the logarithmic fractional parts are the orbit of zero under an irrational rotation of the circle, so everywhere interval frequency under an irrational rotation applies. Thus for $a=2$, digit seven has frequency $(\log 8-\log 7)/\log 10$.

The discrete uniform distribution on a finite set $S$ assigns probability $1/|S|$ to every element of $S$.

An ordered sample from $n$ equally likely values is nondecreasing exactly when it is the unique sorted representative of its multiplicities. Stars and bars counts the nonnegative multiplicities summing to $r$; repeated values are included automatically.

A Bernoulli random variable equals one with probability $p$ and zero with probability $1-p$. Its mean is $p$ and its variance is $p(1-p)$.

If an event occurs with probability $p$ and is independently retained with conditional probability $q$, its retained indicator is Bernoulli with probability $pq$. Applying this independently to a binomial count turns $\mathrm{Binomial}(n,p)$ into $\mathrm{Binomial}(n,pq)$.

A geometric distribution models the number of independent Bernoulli trials needed to obtain the first success. On the convention supported by $1,2,\dots$, its mass is $(1-p{)}^{k-1}p$ and its mean is $1/p$. On the failures-before-first-success convention supported by $0,1,\dots$, its mass is $(1-p{)}^{k}p$ and its mean is $(1-p)/p$. Both conventions have variance $(1-p)/p^2$.

For a positive-integer geometric distribution $P(N=j)=p{q}^{j-1}$ with $q=1-p$, the capped time has masses $p{q}^{j-1}$ for $1\le j<k$ and $q^{k-1}$ at $k$. The last mass includes all later arrivals, not only $N=k$. Its probability generating function and expected value areThe expectation follows either by differentiating the PGF or summing the tail probabilities up to $k$.

When independent uniform draws are made from $n$ types, the time $T_n$ until every type has appeared decomposes as a sum of independent geometric waiting times with parametersConsequentlyand Chebyshev's inequality gives $T_n/(n\log n)\to 1$ in probability.

For uniform independent draws of $n$ coupon types, the waiting times with $r$ unseen types remaining are independent positive-support geometric distributions of parameter $r/n$. Summing their means gives $n{\sum }_{r=1}^{n}1/r$, and summing their variances gives the displayed formula. Independence follows from fresh draws after each discovery and the fact that the next stage's conditional distribution depends only on the count of remaining types.

On the failures-before-the-$r$th-success convention, a negative binomial variable with success probability $p$ has probability mass functionIt has expected value $r(1-p)/p$ and variance $r(1-p)/p^2$.

For the $r$th success to occur after exactly $x$ failures, the final trial must be a success, while the preceding $x+r-1$ trials contain $x$ failures and $r-1$ successes. Choosing the failure locations gives the factor $\left(\genfrac{}{}{0ex}{}{x+r-1}{x}\right)$.

If $Y\mid \Lambda \sim \mathrm{Poisson}(\Lambda )$ and $\Lambda$ has a gamma distribution, then $Y$ has a negative binomial distribution. For gamma shape $\nu$ and scale $\alpha$,

The binomial distribution counts successes in independent Bernoulli trials with common success probability.

For $W\sim \mathrm{Bin}(n,p)$, expand the fourth power of the sum of centered independent Bernoulli variables. Terms with an index occurring once vanish; single-index fourth moments contribute $nq(1-3q)$ and paired second moments contribute $3n(n-1)q^2$. Their sum is the displayed formula. Scaling by $n^4$ yields a uniform $O(n^{-2})$ fourth moment for $W/n-p$.

For independent Bernoulli observations with success probability $p$, the sample proportion is the success count divided by the sample size. It is an unbiased estimator with variance $p(1-p)/n$. It supplies the arm-specific risks in a risk ratio and the mean difference in a two-proportion comparison.

If $x$ successes occur in $n$ independent Bernoulli trials, the binomial likelihood for the success probability $p$ is proportional to $p^x(1-p{)}^{n-x}$.

If a sample of size $r$ is drawn uniformly without replacement from a population of size $n$ containing $c$ marked objects, the number $K$ of marked objects sampled has mass

The multinomial distribution gives the category counts from $n$ independent trials with category probabilities $p_1,\dots ,p_k$.

The vector of category counts is the sum of independent categorical indicator vectors. Their means are $p$ and covariance is $\mathrm{diag}p-p{p}^T$, so the multivariate central limit theorem gives the displayed limit. At uniform probabilities on $k$ categories, Pearson-standardized deviations have covariance equal to the orthogonal projection onto the $(k-1)$-dimensional subspace of zero-sum vectors. Their squared norm has an asymptotic chi-squared distribution with $k-1$ degrees of freedom.

Place $n$ labelled balls independently into $m$ labelled bins, each with probability $1/m$. The vector of bin counts has a multinomial distribution, and each count separately has the binomial distribution $\mathrm{Bin}(n,1/m)$. Distinct bin counts have covariance $-n/m^2$, so they are dependent for $n>0$ and $m>1$. Counts of bins of a specified size are sums of indicator variables; their expectations follow from the linearity of expectation, without any independence between bins.

In a uniform balls-in-bins allocation with $m\to \infty$ and $n/m\to \lambda \in (0,\infty )$, a fixed bin count has convergence in distribution to a Poisson distribution variable of mean $\lambda$ by the Poisson limit theorem. If $N_j$ counts bins containing exactly $j$ balls, symmetry and the linearity of expectation give $\mathbb{E}[N_j]/m=\mathbb{P}(B_1=j)\to e^{-\lambda }{\lambda }^j/j!$ for each fixed $j$. In particular the expected empty fraction tends to $e^{-\lambda }$, and the expected fraction with at least two balls tends to $1-(1+\lambda )e^{-\lambda }$. These are statements about expectations; convergence of the random fractions requires an additional argument.

For category counts $y_1,\dots ,y_k$ with fixed total $n$ and probabilities $p_1,\dots ,p_k$, the multinomial likelihood isIts maximum-likelihood fitted probabilities are ${\hat{p}}_j=y_j/n$.

For counts with a multinomial distribution, this is twice the log likelihood ratio of the saturated cell-probability estimate to fixed probabilities $p_i$. Zero-count terms have limiting value zero. Simulation under the specified probabilities gives a direct reference distribution when asymptotic approximations are unsuitable.

The categorical distribution describes one draw from finitely many categories with probabilities $p_1,\dots ,p_k$.

The Poisson distribution with mean lambda assigns mass $e^{-\lambda }{\lambda }^k/k!$.

A Poisson distribution of parameter $a$ conditioned to lie in $\{0,\dots ,C\}$. It is the occupancy law of a unit-resource loss network. Its mean is the carried load of an Erlang loss resource, $a[1-E(C,a)]$. Differentiating the mean with respect to $\log a$ gives its variance, which is positive for $a>0$ and $C\ge 1$; this proves strict monotonicity of the carried load.

A model with $Y\mid \Lambda \sim \mathrm{Poisson}(\Lambda )$ for a nonnegative random intensity. Conditional expectation and total variance give $\mathbb{E}Y=\mathbb{E}\Lambda$ and $\mathrm{Var}Y=\mathbb{E}\Lambda +\mathrm{Var}\Lambda$, explaining overdispersion. A Poisson-gamma mixture gives the negative binomial distribution.

For a uniform prior distribution on $(l,u)$ with $0<l<u$, conditionally independent Poisson distribution observations have sufficient count $s={\sum }_i{x}_i$. Their Bayesian posterior density is proportional to $t^s{e}^{-nt}$ on that bounded interval. The displayed posterior mean is the Bayes estimator under squared error loss, and is generally not the affine Bühlmann credibility estimate.

When the random intensity is $\alpha +Y$ with $Y$ having an exponential distribution of rate $\beta$, the mixed count is the sum of independent Poisson distribution and support-zero geometric distribution counts. The Poisson parameter is $\alpha$ and the geometric success probability is $\beta /(\beta +1)$. At $\alpha =0$ the law is geometric.

For the shifted-exponential Poisson mixture, the displayed relation holds for $n\ge 2$, initialized by $p_0=e^{-\alpha }\beta /(\beta +1)$ and $p_1=(\alpha +1/(\beta +1))p_0$. It follows by differentiating the probability generating function and comparing coefficients. Nonnegativity follows from the independent Poisson-geometric convolution of independent random variables representation.

A Poisson distribution conditioned on a positive count has probability mass function $\mathbb{P}(Y=y)={\theta }^y/[y!(e^{\theta }-1)]$ for $y\ge 1$ and $\theta >0$.

For the zero-truncated Poisson distribution, $21_{\{Y\text{ even}\}}$ is the unique integrable nonrandomized unbiased estimator of $p=1-e^{-\theta }$. Indeed unbiasedness gives ${\sum }_{y\ge 1}t(y){\theta }^y/y!=e^{\theta }+e^{-\theta }-2$, and power series coefficients force the parity values. Its variance is $2p-p^2$; clipping it to $1_{\{Y\text{ even}\}}$ gives mean squared error $p/2$, strictly smaller for $0<p<1$.

The sum of independent Poisson random variables is Poisson, with parameter equal to the sum of their parameters. This follows by multiplying their probability generating functions $\exp ({\lambda }_i(z-1))$.

For Bernoulli random variables $X_1,\dots ,X_n$, which need not be independent, put $p_i=\mathbb{P}(X_i=1)$, $S={\sum }_i{X}_i$, and $\lambda ={\sum }_i{p}_i$. ThenThe dependence penalty is the total correlation. The proof compares the joint Bernoulli law with the product of Poisson laws of means $p_i$, then applies the data processing inequality for relative entropy to addition.

A Poisson observation model represents a nonnegative count as conditionally Poisson with a mean supplied by an underlying process or regression model.

A Poisson observation is an integer-valued count modeled by a Poisson distribution of specified intensity, possibly conditionally on a latent field. Independent Poisson observations motivate Poisson data fidelity through their log-likelihood.

A zero-inflated Poisson distribution is a mixture of a point mass at zero and a Poisson distribution. If the Poisson component has weight $\pi$ and mean $\lambda$, then its mean is $\pi \lambda$ and its variance is $\pi \lambda +\pi (1-\pi ){\lambda }^2$.

A zero-inflated Poisson regression specifies a structural-zero probability $\pi$ and a susceptible-component Poisson distribution mean $\mu$. In this parametrization,The mean is $(1-\pi )\mu$. Logistic predictors for $\pi$ and logarithmic predictors for $\mu$ allow covariates to affect component membership and count intensity differently. A susceptible individual can still generate a zero count; susceptibility is not equivalent to an observed positive response.