Continuous probability distribution

A continuous probability distribution is described by a density with respect to Lebesgue measure.

For $0<A<B<\infty$, this density makes $\log X$ uniform between $\log A$ and $\log B$. It is a proper finite-interval version of the scale-invariant prior. The chosen logarithm base changes only the endpoint coordinates.

In the shape–minimum parameterization, the density is $\alpha {\beta }^{\alpha }{x}^{-\alpha -1}{1}_{\{x>\beta \}}$, with $\alpha ,\beta >0$. The survival function is $(\beta /x{)}^{\alpha }$ for $x\ge \beta$. Large shape concentrates the distribution near its lower bound. This convention is useful for uniform-Pareto conjugacy.

For nonnegative independent identically distributed summands with polynomial upper tail $\mathbb{P}(X_1\ge x)=x^{-\beta }$ for $x\ge 1$, positivity gives $\mathbb{P}(S_n\ge na)\ge (na{)}^{-\beta }$ for fixed $a>0$ and large $n$. Since this probability is also at most one, its logarithm divided by $n$ tends to zero. No positive exponential moment exists, so the exponential-moment form of the Cramér theorem cannot be applied. The bound proves a logarithmic rate and does not assert an exact tail asymptotic.

At unit scale the probability density function is $r\exp (-r^2/2)$ for $r>0$. Its square has an exponential distribution of rate $1/2$. Thus $\sqrt{-2\log U}$ has this distribution for a uniform distribution variable $U$ on $(0,1)$.

A hypoexponential distribution is the distribution of a sum of independent variables with exponential distributions. For two positive unequal rates $\lambda ,\mu$, convolution of independent random variables gives the probability density functionIt is zero for negative $s$. Its equal-rate limit is the shape-two gamma distribution with rate $\lambda$.

Student's t-distribution with $\nu$ degrees of freedom is the distribution of $Z/\sqrt{V/\nu }$ for independent $Z\sim N(0,1)$ and $V\sim {\chi }_{\nu }^2$.

If $Z\sim N(0,1)$ and $U\sim {\chi }_{\nu }^2$ are independent, then $(Z+\delta )/\sqrt{U/\nu }$ has the noncentral t-distribution with degrees of freedom $\nu$ and noncentrality $\delta$. It describes the test statistic of a Student's t-test under a shifted alternative.

The F-distribution with degrees of freedom $(d_1,d_2)$ is the distribution of $(U_1/d_1)/(U_2/d_2)$ for independent chi-squared random variables $U_1\sim {\chi }_{d_1}^2$ and $U_2\sim {\chi }_{d_2}^2$.

A probability density function is a nonnegative measurable function $f_X$ satisfyingIts integral over the whole sample space is one.

A normalizing constant makes a nonnegative integrable function into a probability density function: if $0<Z=\int h<\infty$, then $h/Z$ is a probability density function.

A density ratio compares two probability density functions relative to the same underlying measure. More generally it is a Radon-Nikodym derivative when one probability measure has absolute continuity of measures with respect to the other.

A probability density is log-concave when its logarithm is a concave function on its support. Tangent lines to a differentiable concave log density lie above its graph, giving exponential envelopes for rejection sampling.

For a probability density function $f$ on $\mathbb{R}$ with Lipschitz bound $L>0$, the triangular lower envelope $f(x+u)\ge (f(x)-L|u|{)}_{+}$ gives $f(x{)}^2\le L$. If $f$ is strictly positive everywhere, the inequality is strict. Equality forces the entire probability density function to coincide with the triangular envelope, which vanishes outside a finite interval.

A joint probability density $f_{X,Y}$ satisfiesIts coordinate integrals are the marginal densities, and integrating it over a rectangle gives the corresponding joint distribution function.

If $Y=g(X)$ for a differentiable bijection, thenThis is the change of variables formula applied to probability density.

The gamma distribution with shape $a>0$ and rate $r>0$ has density

If $X$ and $Y$ are independent random variables with Gamma distributions of shapes $a,b>0$ and the same rate $\theta >0$, then $U=X/(X+Y)$ has the Beta distribution with parameters $(a,b)$, and $V=X+Y$ has the Gamma distribution of shape $a+b$ and rate $\theta$. Moreover $U$ and $V$ are independent. The inverse change of variables $(x,y)=(uv,(1-u)v)$ has absolute Jacobian determinant $v$, which makes the joint probability density factorize on the product domain $0<u<1$, $v>0$. The common rate is essential for this factorization.

If independent random variables $X_i$ have gamma distributions with shapes $a_i$ and one common rate $r$, thenTheir Laplace transforms of nonnegative random variables multiply to $(r/(r+s){)}^{{\sum }_i{a}_i}$, which is the transform of the displayed gamma distribution; uniqueness of the Laplace transform proves the claim.

Writing $\varphi =1/a$, $\theta =-r/a<0$, and $b(\theta )=-\log (-\theta )$ puts the gamma density into exponential dispersion family formIts mean is $a/r$ and its variance is $a/r^2$.

For a gamma random variable of shape $a$ and rate $r$,

An inverse-gamma random variable is the reciprocal of a gamma distribution random variable up to scale. In shape-scale parameters its density is proportional to $x^{-a-1}{e}^{-b/x}$ on $x>0$.

The sum of $n$ independent exponential variables of rate $r$ has the Erlang distribution with densityConvolving with one more exponential proves the formula inductively.

If $m$ is a positive integer and $U_1,\dots ,U_m$ are independent uniform variables, thenhas the gamma distribution with shape $m$ and rate $r$, because each summand is exponential with rate $r$.

The Dirichlet distribution is a probability distribution on vectors $(p_1,\dots ,p_K)$ with nonnegative entries summing to one and density proportional to ${\prod }_k{p}_k^{{\alpha }_k-1}$.

The exponential distribution with rate $c>0$ has densityIt is the gamma distribution with shape one.

For exponential rate $\lambda >0$, the integer part has mass $(1-e^{-\lambda })e^{-\lambda m}$ on nonnegative integers. The fractional part has density $\lambda e^{-\lambda u}/(1-e^{-\lambda })$ for $0\le u<1$. The mixed joint density factors into those marginals, proving independence. The integer part is a geometric distribution on zero-based support, with expectation $1/(e^{\lambda }-1)$.

If $X\sim \mathrm{Exp}(\lambda )$, then direct integration gives

If $T\sim \mathrm{Exp}(\mu )$, thenThus an exponential remaining lifetime has the same law regardless of the elapsed lifetime.

If $A$ and $B$ are independent exponential variables with the same rate, then $A/(A+B)$ is uniform on $(0,1)$ and is independent of $A+B$.

For independent exponential variables $S_i$ of rates $q_i$ and $Q={\sum }_i{q}_i$, let $T={\min }_i{S}_i$ and let $K$ be the minimizing index. ThenConsequently $T$ is exponential with rate $Q$, $\mathbb{P}(K=k)=q_k/Q$, and $K$ and $T$ are independent.

If $N$ is geometric on $\{1,2,\dots \}$ with parameter $p$, independently of unit-rate exponential variables $S_i$, thenis exponential with rate $p$. Its moment-generating function is $p/(p-\theta )$ for $\theta <p$.

A continuous uniform distribution has constant density on its interval of support.

For independent $U,V$ uniform on $[0,1]$, real roots occur on $U\le V^2$, a region of probability $1/3$. On that region both roots are nonpositive. Both lie in $[-1,0]$ exactly when $\max (0,2V-1)\le U\le V^2$. Its area is $1/12$, so the conditional probability of both absolute values being at most one is $1/4$. Joint probability density integration turns algebraic discriminant and root constraints into simple planar areas.

A uniform random variable has a uniform distribution. Independent such variables can supply independent inverse transform sampling; reusing a single variable for two transforms generally creates dependence.

A Steinhaus random variable is uniformly distributed on the complex unit circle. It can be generated as $ϵ\cos \theta +i\delta \sin \theta$ using independent signs $ϵ,\delta$ and an independent uniform angle $\theta \in [0,\pi /2]$. This conditional sign representation transfers Rademacher sum inequalities to random complex phases.

For independent Steinhaus random variables, condition on their acute angles and use the sharp Rademacher second-moment inequality on the $2d$ sign coefficients $a_i\cos {\theta }_i,i{a}_i\sin {\theta }_i$. Their squared moduli sum to ${\sum }_i|a_i{|}^2$ independently of the angles, so the conditional first moment is at least the same fixed square-root bound. Averaging proves the assertion.

The square of a uniform distribution variable on $(0,1)$ has cumulative distribution function $\sqrt{x}$ and an integrable inverse-square-root probability density function. Reflection replaces $x$ by $1-x$ in that density.

For independent $U_1,U_2\sim \mathrm{Uniform}(0,1)$, the sum has triangular densityIts upper tail for $1\le c\le 2$ is $(2-c{)}^2/2$.

Let $X\ge 0$ have density $g$, let $U$ be independent and uniform on $[0,1]$, and setThen

Write $h(y)={\int }_y^{\infty }g(t)/tdt$. The two pieces in a uniform split are independent exactly whenSetting $z=0$ shows that $h(y)=c{e}^{-cy}$ for some $c>0$. Thus the pieces are independent exactly when they are independent exponential variables of the same rate; equivalently, the total has gamma density $c^{2}x{e}^{-cx}$.

The centred Laplace distribution with rate $\theta$ has density $\theta e^{-\theta |x|}/2$.

Independent centered Laplace distributions for pixel intensities give the prior distributionThis prior favors small intensities through its sharp peak at zero, while its tails permit isolated appreciable pixels. Its negative log density is an L1 norm penalty, so with a Gaussian likelihood a maximum a posteriori estimate minimizes a sum of squared residuals plus that penalty. A draw from this continuous prior has no exactly zero coordinate with positive probability; sparsity of a maximizing estimate and concentration of prior mass near zero are different statements. The product prior also does not encode spatial contiguity. Inverse transform sampling produces each coordinate from an independent uniform distribution by ${\lambda }^{-1}\log (2U)$ for $U\le 1/2$ and $-{\lambda }^{-1}\log (2(1-U))$ otherwise, ignoring the probability-zero endpoints.

In the modelmaximum likelihood minimizes the sum of absolute residualsThe scale estimate is $\hat{\sigma }=S(\hat{\beta })/n$.