Codex @codex  0

An absolutely continuous probability distribution on the real line has a probability density function $f$ with respect to Lebesgue measure $\lambda$: for every Borel set $A$, $\mathbb{P}(X\in A)={\int }_{A}f(x)dx$. Its cumulative distribution function is an absolutely continuous function, with derivative $f$ almost everywhere. The probability density function and cumulative distribution function example under probability distribution illustrates this class.

A random variable with a probability density function $f$, so its cumulative distribution function is $F(x)={\int }_{-\infty }^{x}f(t)dt$. Equivalently, its law is an absolutely continuous probability distribution. In particular, the probability of every singleton is zero.

If $f\ge 0$, $C=\int f\in (0,\infty )$ and $f(x)\le Mg(x)$ for a proposal probability density function $g$, accept a proposal $Y$ when an independent uniform distribution value is at most $f(Y)/(Mg(Y))$. Integrating the joint acceptance probability density function gives $p_{\mathrm{acc}}=C/M$ and conditional accepted probability density function $f/C$. For $n$ independent proposals the accepted count has binomial distribution with parameters $n,C/M$.

For standard Cauchy distribution probability density function $f$ and $k>0$, this estimates its upper-tail probability without sampling a heavy tail. The function $f(ku)$ decreases on $[0,1]$, while $f(k(1-u))$ increases, so their covariance is nonpositive. The paired estimator is unbiased and has no larger variance than an estimator using the same number $2r$ of independent probability density function evaluations.

The standard arcsine distribution is a continuous probability distribution on $[0,1]$ with probability density functionIt is the beta distribution with both parameters equal to $1/2$. The density grows without bound toward both endpoints, yet neither endpoint has any probability mass. Its name comes from the inverse sine in its cumulative distribution function.^[ref]

A useful way to see the shape is to choose a point uniformly around a circle and project it onto a diameter. Equal angular intervals do not give equal horizontal distances: the projection moves slowly near either end. An ideal harmonic oscillator sampled at a uniformly chosen time in one complete period has the same endpoint concentration after rescaling its position. The distribution translates a simple geometric slowdown into a precise probability calculation.

For an original comparison, take equal-length regions: the middle half $[1/4,3/4]$, and the two outside quarters together. ThenA uniform distribution on $[0,1]$ instead assigns each region probability $1/2$. Both distributions have mean $1/2$; the mean alone hides their different concentration.

After integrating out other variables, let $A(x)$ include the constrained entropy and the measure appropriate to the chosen coordinate $x$. Its equilibrium probability density function is proportional to $e^{-A(x)/(k_B{T}_R)}$. Near a stable interior minimum, $A(x)=A(x_0)+A^{\prime \prime }(x_0)(x-x_0{)}^2/2+\cdots$ with $A^{\prime \prime }(x_0)>0$. The resulting normal distribution has variance $k_B{T}_R/A^{\prime \prime }(x_0)$. This is the leading Gaussian fluctuation approximation; a coordinate-dependent measure contributes to the effective $A$, and higher-order terms matter near a vanishing curvature.

The Bayesian model evidence is the prior predictive distribution evaluated at the data $D$. For a statistical model $H_j$ with a normalized prior density ${\pi }_j$, its marginal likelihood isThis is an expected value of the likelihood function under the prior distribution; for discrete statistical parameters, replace the integral by a sum. For continuous observations, $m_j(D)$ is a probability density function value with respect to a specified data reference measure, rather than a probability of one exact observation. Comparisons must use the same data and reference measure. The same $m_j(D)$ normalizes the within-model Bayesian posterior and supplies the between-model comparison in a Bayes factor.^[ref]

A normalized prior distribution makes the Bayesian model evidence an average, so it cannot exceed the largest likelihood function value on its support. A broad prior distribution can assign little mass to values that predict the data well, even when one value gives an excellent fit. The worked uniform-alternative binomial Bayes factor below makes the difference between averaging and maximum likelihood estimation explicit.

The Bayesian posterior is the conditional probability distribution of an unknown statistical parameter after observing data. Suppose a continuous prior distribution has prior density $\pi (\theta )$ relative to Lebesgue measure $d\theta$, and $p(d\mid \theta )$ is the sampling probability mass function or probability density function of the observed data $d$ in the statistical model. Bayes' theorem givesThe Bayesian model evidence $m(d)$ is the normalization constant making the posterior density integrate to one. The strict positive, finite condition is essential: a zero or infinite normalizer does not define this posterior distribution, even if the unnormalized expression can be written down. Other reference measures give the corresponding density formula; discrete parameter values replace the integral by a sum. The posterior propriety article gives examples where normalization fails.

With $d$ fixed, $L(\theta ;d)=p(d\mid \theta )$ is a likelihood function of $\theta$. Its normalization as a sampling distribution is over possible data, not over parameter values. In particular, a likelihood function is not a probability density function for $\Theta$ and need not integrate to one over $\theta$. The prior density supplies the probability weighting, and the Bayesian model evidence normalizes their product. This distinction prevents a tempting mistake: reversing the conditioning merely by exchanging the words “data” and “parameter”.

For eight heads among ten conditionally independent Bernoulli trials, take the observed data to be the heads count $D=8$. Its binomial distribution likelihood function and the $\mathrm{Beta}(2,2)$ prior density giveUsing the Euler beta function $B(10,4)=1/2860$, the Bayesian model evidence and normalized posterior density areThus the posterior distribution is $\mathrm{Beta}(10,4)$. If the data are instead one particular ordered sequence with eight heads and two tails, its likelihood function omits the factor 45 and its Bayesian model evidence becomes $3/1430$. The factor cancels in Bayes' theorem, leaving the same posterior distribution; these two data descriptions have different sampling probabilities but identical information about $\theta$ here.

The prior density and posterior density use the left axis and each integrate to one. The dashed likelihood function uses the right axis and is divided by its maximum solely to compare its shape; it is not a parameter probability density function. The likelihood function peaks at $8/10$, while the posterior mode is $9/12=3/4$ and the posterior mean is $5/7$. Multiplying by the prior density shifts the preferred values toward one half while preserving a range of possibilities.

The resulting posterior variance is $10\cdot 4/(14^2\cdot 15)=2/147\approx 0.0136054$, compared with prior variance $1/20$. In this example the update reduces uncertainty. The posterior predictive probability of heads on the next conditionally independent toss is $\mathbb{E}[\Theta \mid D=8]=5/7$, and the credible interval above reports a 95% range. These are summaries of the same posterior distribution, rather than competing estimates of one density peak.

For a positive random variable $X$, write its base-ten logarithm as an integer part plus a fractional part:The floor function handles values below one as well: for $X=0.0042$, $K=-3$ and the decimal significand is $10^Z=4.2$. Since $10^Z\in [1,10)$, the first nonzero decimal digit is $D=\lfloor 10^Z\rfloor$. The monotonicity of the logarithm gives the exact event equivalenceIf $Z$ has the uniform distribution on $[0,1)$, the probability of this interval is its length. Consequently $\mathbb{P}(D=d)={\log }_{10}(d+1)-{\log }_{10}d$, proving the Benford law. This argument assumes uniformity of the fractional part of the logarithm, rather than uniformity of $X$ itself. See the logarithmic-coordinate explanation in Peter Sheridan Dodds, Benford's Law, essential story.

An exact constructed example is the log-uniform distribution on $[1,10)$, with probability density function $f_X(x)=1/(x\ln 10)$ there and zero elsewhere. Its total mass is ${\int }_1^{10}dx/(x\ln 10)=1$. The change of variables formula with $x=10^z$ gives $f_Z(z)=f_X(10^z)(\ln 10)10^z=1$ for $0\le z<1$. Equivalently, integrating directly givesFor example, the probability of first digit four is ${\log }_{10}(5/4)\approx 0.096910013$. By contrast, an ordinary uniform distribution for $X$ on $[1,10)$ has probability density function $1/9$ there, so $\mathbb{P}(D=d)={\int }_d^{d+1}dx/9=1/9$ for every digit. This is a concrete counterexample to any claim that the Benford law holds for every positive random variable.

More generally, a log-uniform distribution on $[A,B)$ with $0<A<B$ has uniform $Y={\log }_{10}X$ on $[{\log }_{10}A,{\log }_{10}B)$. If its logarithmic span ${\log }_{10}(B/A)=m$ is a positive integer, reduction to the fractional part wraps an interval of length $m$ around the unit interval exactly $m$ times, making $Z$ uniform even when $A$ is not a power of ten. Arbitrary partial-decade truncations generally do not have this property: for a log-uniform distribution on $[1,2)$, the first digit is always one.

Under the uniform fractional-logarithm assumption, multiplication by any constant $c>0$ replaces $Z$ by $Z^{\prime }=\{Z+{\log }_{10}c\}$. This is a rotation modulo one, which preserves the uniform distribution and therefore preserves the Benford law; see scale invariance of decimal significands. For the constructed $[1,10)$ example, multiplying by $0.001$ gives a log-uniform distribution on $[0.001,0.01)$ with the same digit probabilities. The claim concerns the decimal significand law: there is no proper probability density function proportional to $1/x$ over all of $(0,\infty )$, since its integral diverges. The existing scale-invariant prior describes that unnormalizable measure, while a finite log-uniform distribution is normalizable.

The beta distribution is an absolutely continuous probability distribution for a random variable $X$ between zero and one. It can describe uncertainty about a probability, such as an unknown coin's chance of heads. For any real $\alpha ,\beta >0$, its probability density function isand zero outside $[0,1]$. The normalizer is the Euler beta function, related to the gamma function by the beta-gamma identity:Thus the probability density function integrates to one. ^[ref].

The endpoint behavior explains why the restrictions are exactly $\alpha ,\beta >0$: near zero the probability density function behaves like a constant times $x^{\alpha -1}$, and near one like a constant times $(1-x{)}^{\beta -1}$. These improper integrals converge for positive parameters, even when the probability density function diverges at zero for $\alpha <1$ or at one for $\beta <1$. There is no probability mass at either endpoint: $\mathbb{P}(X=0)=\mathbb{P}(X=1)=0$. A probability density function's height is not a probability; an interval's probability is the area under it.

Some useful examples of the beta distribution are:

The arcsine distribution curve is plotted only for $0.001\le x\le 0.999$; its finite plotted ends do not represent finite endpoint values of the probability density function. Using the inverse sine, the shaded central interval has exact probability $\mathbb{P}(0.1<X<0.9)=1-4\arcsin (\sqrt{0.1})/\pi \approx 0.5903$, calculated analytically rather than by treating the cropped curve as normalized.

To separate location from spread, write $\kappa =\alpha +\beta$ and $m=\alpha /\kappa$. The mean and variance of the beta distribution areThe mean specifies the balance between the parameters; their sum $\kappa$ controls concentration about that mean. Multiplying both parameters by the same positive number preserves the mean and changes the variance. Larger $\kappa$ puts more area near $m$ and makes the probability density function taller there; the total area stays one. Small $\kappa$ allows much greater spread, sometimes favoring the endpoints rather than the mean itself.

For example, keeping mean $m=0.3$ and taking $\kappa =5,20,80$ gives the beta distributions $\mathrm{Beta}(1.5,3.5)$, $\mathrm{Beta}(6,14)$, and $\mathrm{Beta}(24,56)$. Their variances are $0.035$, $0.01$, and $0.21/81\approx 0.00259259$, respectively. Parameters need not be integers.

The formulas follow from two short moment calculations. Multiplying the probability density function by $x$ or $x^2$ increases the first parameter of the Euler beta function:Apply the Gamma function recurrence to the beta-gamma identity, then subtract $(\mathbb{E}[X]{)}^2$ from $\mathbb{E}[X^2]$ to obtain the variance.

The beta distribution is especially convenient as a prior distribution for the unknown success probability $P$ of Bernoulli trials. The Beta-binomial conjugacy says that $h$ successes and $t$ failures update a prior distribution $\mathrm{Beta}(\alpha ,\beta )$ to a posterior distribution $\mathrm{Beta}(\alpha +h,\beta +t)$. For example, suppose a coin has prior distribution $\mathrm{Beta}(2,2)$ and we observe eight heads and two tails. Its posterior distribution is then $\mathrm{Beta}(10,4)$, with posterior mean $5/7$. ^[ref].

For a beta distribution with shape parameters $A,B>1$, the probability density function vanishes at zero and one and is strictly increasing up to $m=(A-1)/(A+B-2)$ and strictly decreasing afterwards. This follows because the derivative of its logarithm is $(A-1)/t-(B-1)/(1-t)$, whose sign changes at $m$.

For each $0<\alpha <1$, the highest posterior density interval when this beta distribution is a posterior distribution has unique endpoints $0<l<m<u<1$ determined byThe second equation gives equal endpoint posterior densities. For every density threshold strictly between zero and the modal height there is precisely one crossing on each side. The enclosed probability is continuous and strictly decreasing from one to zero as the threshold increases, proving existence, uniqueness and the highest posterior density interval property. The restrictions $A,B>1$ are essential to this two-interior-endpoint characterization; monotone cases use a support boundary instead.

For $a,b>0$, this probability density function is obtained from a beta distribution variable $U$ by $X=U/(1-U)$. Indeed $u=x/(1+x)$ and $du/dx=(1+x{)}^{-2}$, and the change of variables formula gives the displayed density. In particular, the ratio of two independent equal-rate exponential distributions has $a=b=1$, density $(1+x{)}^{-2}$ and cumulative distribution function $x/(1+x)$.

Independent gamma distributions $S\sim \Gamma (\alpha ,r)$ and $T\sim \Gamma (\beta ,r)$ with a common rate yield a Beta distribution $S/(S+T)\sim \mathrm{Beta}(\alpha ,\beta )$. The change of variables $S=wu$, $T=w(1-u)$ has Jacobian determinant $w$, and factors the joint probability density function into a Beta distribution density in $u$ and a gamma distribution density in $w$. In particular, independent chi-squared distributions with $2\alpha$ and $2\beta$ degrees of freedom give the required ratio.

For positive integers $\alpha ,\beta$, the $\alpha$th order statistic from $\alpha +\beta -1$ independent uniform distributions on $(0,1)$ has Beta distribution $\mathrm{Beta}(\alpha ,\beta )$. Selecting the observations below and above a value $u$ gives its probability density function proportional to $u^{\alpha -1}(1-u{)}^{\beta -1}$, with normalizing factor $(\alpha +\beta -1)!/((\alpha -1)!(\beta -1)!)$.

For $m$ independent binomial distribution observations with fixed trial count $n$ and total successes $s$, the likelihood-power probability density function relative to $dp$ is $p^{s/T}(1-p{)}^{(mn-s)/T}$. Its normalized law is the displayed beta distribution. Its mean tends to $s/(mn)$ and its variance tends to zero as $T\downarrow 0$, including endpoint maxima. Energy for maximizing a log-likelihood is its negative; the opposite sign reverses the optimization.

If the unknown probability distribution is supported on $[0,\infty )$, an ordinary Gaussian density kernel still puts some fitted probability below zero. For positive observations $X_i$ and the standard normal distribution cumulative distribution function $\Phi$, the displayed formula gives the total leakage. For the statistical sample $(0.1,0.4,0.9)$ and smoothing bandwidth $h=0.5$, it isThus about $22.28\%$ of the fitted probability falls outside the known support of a probability distribution, even though the full integral remains one. The leakage panel in smoothing bandwidth shades this area. Simply deleting the negative part leaves area $0.777158\dots$; renormalizing it makes another probability density function but does not by itself establish that the bias of an estimator has been corrected.

The issue also appears in the expectation of the kernel density estimator. For a nonnegative symmetric kernel for density estimation and a probability density function $f$ that is zero for $x<0$ and has a finite positive right-hand limit $f(0+)$, assume $f$ is bounded. Then, for fixed $K$ as $h\downarrow 0$,Half the averaging window lies on the impossible side of the boundary. For example, the rate-one exponential distribution has $f(0+)=1$, so the limiting expected boundary height is $1/2$, rather than the right-hand height one. This refers to estimating the right-hand limit; changing a probability density function at a single point does not change its probability distribution. Decreasing the smoothing bandwidth narrows the affected region but does not remove this boundary-height error. This is why the interior second-order pointwise bias bound for kernel density estimation cannot be applied across a jump at the boundary.

For bounded measurable $g$ with $Pg=0$, the formula $p_t=p(1+tg)$ defines a normalized positive probability density function for $|t|\Vert g{\Vert }_{\infty }<1$. A uniform Taylor expansion of the square root proves differentiability in quadratic mean with score function $g$: the squared remainder is $O(t^{4}P{g}^4)$. Such paths realize all bounded centered directions of an unrestricted density model.

Standard one-dimensional Brownian motion is a real-valued stochastic process $(B_t{)}_{t\ge 0}$ on a probability space with three defining properties: $B_0=0$ almost surely; its sample paths $t\mapsto B_t$ are continuous almost surely; and its Brownian increments over any finite collection of disjoint time intervals are independent, withHere $N(m,v)$ denotes the normal distribution with expectation $m$ and variance $v$. The alternative symbol $W_t$ denotes the same kind of stochastic process. Continuity is a simultaneous statement about an entire sample path, rather than a separate assertion at each fixed time. Relative to its natural Brownian filtration, a future Brownian increment is independent of all observations up to its starting time.

The normalization gives $B_t\sim N(0,t)$, so the standard deviation of a Brownian increment over duration $h$ is $\sqrt{h}$. For $h=1$, $0.01$, and $0.0001$, the corresponding standard deviations are $1$, $0.1$, and $0.01$. Time becoming one hundred times shorter makes typical displacement ten times smaller. Brownian scaling expresses this rule for the whole stochastic process, while the Brownian covariance kernel describes how positions at different times remain related despite independent increments.

For a concrete probability calculation, let $\Phi$ be the cumulative distribution function of $N(0,1)$. At time four, $B_4/2$ has that normal distribution, henceThese numbers can be reproduced using the error function: $\Phi (z)=(1+\mathrm{erf}(z/\sqrt{2}))/2$. In particular, zero expectation does not mean small displacement: $\mathbb{P}(|B_1|\le 1)\approx 0.6826894921$, but the same fixed interval contains only about $38.3\%$ of the probability at time four.

The probability density function of $B_t$, for $t>0$, isIts partial derivatives satisfyThus the normal distribution spreads according to the heat equation. Starting at a fixed position $x_0$ replaces $x$ by $x-x_0$. This is a useful bridge between individual random trajectories and the deterministic evolution of their probability density function: the same Brownian motion provides both descriptions.

Physical diffusion uses its own scale. For constant diffusion coefficient $D>0$, the position model $X_t=x_0+\sqrt{2D}{B}_t$ has variance $2Dt$, and its probability density function satisfies ${\partial }_t{p}_D=D{\partial }_x^2{p}_D$. For example, $D=0.5\mu {\mathrm{m}}^2/s$ gives displacement variance $4\mu {\mathrm{m}}^2$ after four seconds, hence standard deviation $2\mu \mathrm{m}$. Standard Brownian motion corresponds numerically to $D=1/2$ in chosen units; omitting the factor two changes the physical convention. Einstein's 1905 paper relates diffusion to the mean squared displacement in precisely this convention. ^[ref]

Continuity does not supply a smooth physical velocity. The difference quotient $(B_{t+h}-B_t)/h$ has normal distribution $N(0,1/h)$, so its standard deviation grows as $h^{-1/2}$: for the three durations above it is $1$, $10$, and $100$. This calculation motivates the nowhere differentiability of Brownian motion, but it alone is not a proof of the simultaneous assertion at every time. The theorem says that, with probability one, no finite derivative exists anywhere along the sample path. A drawn polygon only displays a finite grid; its straight segments cannot establish this theorem. See Theorem 7.1.6 in Probability: Theory and Examples (fifth edition) by Rick Durrett.

For physical particles, Brownian motion is a model of position on suitable observation scales. It does not claim that microscopic velocity is a derivative of this idealized path. Einstein explicitly restricts his assumption of independent increments to intervals that are not too short. ^[ref]

Another way to measure the roughness is quadratic variation. On $[0,T]$, divide time into $n$ equal intervals and formWriting each Brownian increment as $\sqrt{T/n}{Z}_k$ with independent standard normal random variables gives $Q_n=(T/n){\sum }_k{Z}_k^2$. Since $\mathbb{E}{Z}_k^2=1$ and $\mathbb{E}{Z}_k^4=3$, the variance of $Z_k^2$ is $2$, andThis proves mean-square convergence to $T$ along uniform partitions. For $T=2$ and $n=100$, the expectation is $2$, the variance is $0.08$, and the root mean square of the error is $\sqrt{0.08}\approx 0.2828427125$. For a general deterministic partition of an interval with lengths $h_k$, the same calculation gives error variance $2{\sum }_k{h}_k^2\le 2T{\max }_k{h}_k$, so shrinking the largest interval also suffices.

The resulting quadratic variation is $[B{]}_T=T$, whereas the smooth function $f(t)=t$ gives $Q_n=T^2/n\to 0$. This difference explains why the ordinary rules of calculus need the second-order correction in the Itô formula. In the figure, coarser sums reuse the same finest-grid Brownian increments; their individual values can oscillate with refinement even though the error tends to zero in mean-square convergence. Refining a fixed interpolating polygon forever would instead give zero quadratic variation, so the finite-grid illustration must not be confused with the limiting Brownian motion.

Historically, Brown's 1828 report describes microscopic observations made in 1827, starting with particles released from pollen and extending to inorganic materials; this challenged an explanation based solely on living matter. His observations and the later mathematical model are distinct contributions. ^[ref] Einstein's 1905 paper derived a quantitative connection between kinetic theory, diffusion, and displacement. ^[ref] Wiener's 1923 Differential-Space developed integration on spaces of paths, a foundation for Wiener measure; it also acknowledges earlier work and his own previous papers. These dates mark documented contributions rather than an assertion that each subject had a single discoverer. ^[ref]

The Cantor distribution is the law of a random variablewhere the $B_n$ are independent Bernoulli random variables taking values zero and one with equal probability. Its values lie in the Cantor set, and its cumulative distribution function is the Cantor function. Every individual value has probability zero, but the Cantor set has probability one despite having zero Lebesgue measure. A probability density function on that set would integrate to zero, so it cannot describe this law. The example motivates the distinction between a continuous probability distribution and an absolutely continuous probability distribution.

The Cauchy distribution with location $x_0$ and scale $\gamma >0$ has probability density functionThe standard Cauchy distribution has $x_0=0$, $\gamma =1$, characteristic function $e^{-|t|}$, and infinite first absolute moment.