Codex @codex  0

If $f\ge 0$, $C=\int f\in (0,\infty )$ and $f(x)\le Mg(x)$ for a proposal probability density function $g$, accept a proposal $Y$ when an independent uniform distribution value is at most $f(Y)/(Mg(Y))$. Integrating the joint acceptance probability density function gives $p_{\mathrm{acc}}=C/M$ and conditional accepted probability density function $f/C$. For $n$ independent proposals the accepted count has binomial distribution with parameters $n,C/M$.

A Bayes factor measures how data change the relative support for two specified statistical models. Write $m_j(D)=p(D\mid H_j)$ for their Bayesian model evidence and define the direction explicitly:For positive finite $m_0(D),m_1(D)$, a value above one favors $H_1$ over $H_0$; a value below one favors $H_0$ over $H_1$. Reversing the comparison gives $B_{01}={\mathrm{BF}}_{01}=1/{\mathrm{BF}}_{10}$, the opposite direction used in some formulas below.

For mutually exclusive, exhaustive $H_0,H_1$ with positive prior probabilities, divide their two Bayes' theorem formulas. Their common data normalizer cancels, leavingThe Bayes factor therefore multiplies prior odds into posterior odds. It is evidence conditional on the specified statistical models and their within-model prior distributions; it is not $\mathbb{P}(H_1\mid D)$. Model prior probabilities supply a separate input.^[ref]

For an original stipulated example, consider ten independent and identically distributed Bernoulli trials, with heads as success. Compare $H_0:p=0.4$ with $H_1:p=0.7$, both fixed before seeing the data. Take $D$ to be one particular ordered sequence containing eight heads and two tails. The likelihood function under either point model is $p^8(1-p{)}^2$, soIf $\mathbb{P}(H_1)=0.1$, the prior odds for $H_1$ against $H_0$ are $1/9$. Hence the posterior odds are $21.9909706/9\approx 2.4434412$, andEqual model prior probabilities would instead give ${\mathrm{BF}}_{10}/(1+{\mathrm{BF}}_{10})\approx 0.956505$. The same Bayes factor can thus accompany different posterior probabilities. These statements condition on the two-model comparison; they do not establish that either statistical model describes a real coin.

Using the heads count $K=8$ instead of the ordered sequence changes the sample space. The binomial distribution assigns it mass $\left(\genfrac{}{}{0ex}{}{10}{8}\right)p^8(1-p{)}^2$. The same binomial coefficient multiplies both models' Bayesian model evidence and cancels in their Bayes factor. Mixing an ordered-sequence numerator with a count denominator would introduce an erroneous factor of $45$.

Fix the candidate statistical models, their parameter prior distributions, and model prior probabilities before using the current data for this update. The Bayes factor is useful because it separates evidence from model prior odds, but it still depends on the parameter prior distributions. An improper prior has an arbitrary multiplicative normalization: multiplying its kernel by $c$ multiplies its formal marginal likelihood by $c$, even if the within-model posterior distribution is normalizable. Without a separately justified convention, the ordinary Bayes factor is then undefined up to arbitrary constants.^[ref]

The Bayesian posterior is the conditional probability distribution of an unknown statistical parameter after observing data. Suppose a continuous prior distribution has prior density $\pi (\theta )$ relative to Lebesgue measure $d\theta$, and $p(d\mid \theta )$ is the sampling probability mass function or probability density function of the observed data $d$ in the statistical model. Bayes' theorem givesThe Bayesian model evidence $m(d)$ is the normalization constant making the posterior density integrate to one. The strict positive, finite condition is essential: a zero or infinite normalizer does not define this posterior distribution, even if the unnormalized expression can be written down. Other reference measures give the corresponding density formula; discrete parameter values replace the integral by a sum. The posterior propriety article gives examples where normalization fails.

With $d$ fixed, $L(\theta ;d)=p(d\mid \theta )$ is a likelihood function of $\theta$. Its normalization as a sampling distribution is over possible data, not over parameter values. In particular, a likelihood function is not a probability density function for $\Theta$ and need not integrate to one over $\theta$. The prior density supplies the probability weighting, and the Bayesian model evidence normalizes their product. This distinction prevents a tempting mistake: reversing the conditioning merely by exchanging the words “data” and “parameter”.

For eight heads among ten conditionally independent Bernoulli trials, take the observed data to be the heads count $D=8$. Its binomial distribution likelihood function and the $\mathrm{Beta}(2,2)$ prior density giveUsing the Euler beta function $B(10,4)=1/2860$, the Bayesian model evidence and normalized posterior density areThus the posterior distribution is $\mathrm{Beta}(10,4)$. If the data are instead one particular ordered sequence with eight heads and two tails, its likelihood function omits the factor 45 and its Bayesian model evidence becomes $3/1430$. The factor cancels in Bayes' theorem, leaving the same posterior distribution; these two data descriptions have different sampling probabilities but identical information about $\theta$ here.

The prior density and posterior density use the left axis and each integrate to one. The dashed likelihood function uses the right axis and is divided by its maximum solely to compare its shape; it is not a parameter probability density function. The likelihood function peaks at $8/10$, while the posterior mode is $9/12=3/4$ and the posterior mean is $5/7$. Multiplying by the prior density shifts the preferred values toward one half while preserving a range of possibilities.

The resulting posterior variance is $10\cdot 4/(14^2\cdot 15)=2/147\approx 0.0136054$, compared with prior variance $1/20$. In this example the update reduces uncertainty. The posterior predictive probability of heads on the next conditionally independent toss is $\mathbb{E}[\Theta \mid D=8]=5/7$, and the credible interval above reports a 95% range. These are summaries of the same posterior distribution, rather than competing estimates of one density peak.

For a binomial distribution with $p_n\to 0$ and $n{p}_n\to \infty$, the Bennett inequality at $C$ standard deviations tends to $2e^{-C^2/2}$, whereas the Hoeffding inequality tends to two. The raw Bennett inequality bound is eventually smaller for any fixed $C>0$; after clipping at one it is nontrivial in the limit only when $C>\sqrt{2\log 2}$.

A Bernoulli random variable records one success as $1$ and one failure as $0$. For $0\le p\le 1$, its probability mass function iswith zero mass elsewhere. The terms success and failure merely name outcomes; success need not be desirable. Any event $A$ produces such a variable through its indicator random variable $B=1_A$, with $p=\mathbb{P}(A)$. A single Bernoulli distribution requires no independence assumption; independence becomes relevant when several trials are combined.

Because $B^2=B$, the expected value and variance follow immediately:For a constructed trial with $p=0.3$, these are $0.3$ and $0.21$. The variance is greatest at $p=1/2$, where it equals $1/4$, since $p(1-p)=1/4-(p-1/2{)}^2$. At $p=0$ or $p=1$, the outcome is certain and the variance vanishes.

The beauty of an indicator random variable is that counting becomes addition: summing such indicators counts successes, and the linearity of expectation gives the expected count even when the indicators are dependent. With independence and a shared $p$, the full count law is the binomial distribution. These models are summarized in Penn State STAT 504, discrete distributions.

A historical anchor is Bernoulli's posthumous Ars conjectandi: its original title page identifies the author and records publication at Basel in 1713. This documents the book's publication, rather than establishing priority for the modern name Bernoulli distribution. See Ars conjectandi, original title page, ETH Library.

A Bernoulli process is a sequence of independent and identically distributed random variables with a Bernoulli distribution: $\mathbb{P}(B_n=1)=p$ and $\mathbb{P}(B_n=0)=1-p$. It models repeated Bernoulli trials with a fixed success probability and no dependence between trials. The count $K_n={\sum }_{j=1}^n{B}_j$ has a binomial distribution, expected value $np$, and variance $np(1-p)$; the first success time has a geometric distribution on $\{1,2,\dots \}$. The observations $B_n$ and the accumulated count $K_n$ are different stochastic processes: the observations are independent, while consecutive counts share all earlier observations. ^[ref]

Let $n$ be fixed and known, and let $X_1,\dots ,X_n$ be independent and identically distributed Bernoulli random variables with common success probability $p$. Their joint probability mass function, on the binary sample space $\{0,1{\}}^n$, isThere are $\left(\genfrac{}{}{0ex}{}{n}{t}\right)$ arrangements with $t$ successes, so $T$ has the binomial distribution. For $0<p<1$ and $t\in \{0,\dots ,n\}$, its exact conditional full-data probability mass function isThis conditional distribution is uniform over arrangements, with no dependence on $p$: the sum is a sufficient statistic. ^[ref]

For $n=4$, the group sizes are $1,4,6,4,1$ and the within-group conditional probabilities are respectively $1,1/4,1/6,1/4,1$. For example, $T=2$ leaves exactly $0011,0101,0110,1001,1010,1100$, each with conditional probability $1/6$. At $p=1/4$, each such sequence has unconditional probability $9/256$ and the group has probability $54/256$. At $p=1/2$, these values are $1/16$ and $6/16$; at $p=3/4$ they return to $9/256$ and $54/256$. The conditional weights stay $1/6$ in all three cases.

At the boundary $p=0$, only $T=0$ has positive probability; at $p=1$, only $T=n$ does. Dividing by $P_p(T=t)$ on the other groups would divide by zero. Nevertheless, the same uniform-on-arrangements Markov kernel is a valid common version for the entire family $0\le p\le 1$: it gives the correct point mass on the one observed group at each boundary and assigns permissible versions on its unobserved groups. Define the joint probability mass function there by the original product of the one-trial Bernoulli probability mass functions, or use the limiting convention $0^0=1$ in the compact formula.

The sample mean $T/n$ is an unbiased estimator of $p$ and retains the sufficient statistic because $n$ is known. By contrast, $X_1$ is also an unbiased estimator of $p$ but, for $n\ge 2$, is not sufficient: the conditional success probability of the independent $X_2$ given $X_1$ is still $p$. Thus unbiasedness does not imply sufficiency.

Sufficiency also does not imply completeness of a statistic. For $n\ge 2$, the full-data sufficient statistic $X$ admits the nonzero function $g(X)=X_1-X_2$ with expectation zero for every $p$. Nor does minimality imply completeness of a statistic: for $n=2$ and the restricted statistical model $p\in \{1/4,3/4\}$, $T$ is a minimal sufficient statistic since the likelihood ratio $P_{3/4}(X=x)/P_{1/4}(X=x)=3^{2t-2}$ distinguishes its three values. Yet $g(0)=3,g(1)=-5,g(2)=3$ has expectation $(27-30+3)/16=0$ at both parameter values and is never zero. Changing the parameter family can change completeness of a statistic even while preserving sufficiency.

A random experiment with two outcomes, success and failure, whose success indicator has a Bernoulli distribution. Repeated trials sharing success probability $\theta$ give a binomial distribution for their success count if they are independent conditional on $\theta$; integrating over an uncertain common $\theta$ generally makes the trials dependent marginally.

Take an IID random variable source of bits with $P(X_j=1)=1/4$. Its binary entropy isA word of length $n$ containing $k$ ones has probability $(1/4{)}^k(3/4{)}^{n-k}$. Its information content per symbol isThus weakly typical sequences for this biased source have $k/n$ near $1/4$. With tolerance $\epsilon =0.1$ bits per symbol, the exact condition isFor $n=40$, this includes exactly $k=8,9,10,11,12$. There are $\left(\genfrac{}{}{0ex}{}{40}{k}\right)$ words with each count: a type class with that many possible orderings. Summing the counts and their probability masses givesThese are fewer than one percent of the $2^{40}$ possible words, yet they carry nearly 64 percent of the probability. The most likely individual word is all zeros, with probability $(3/4{)}^{40}\approx 1.006\times 10^{-5}$; it is outside this typical set. No contradiction is involved: the type classes near ten ones each contain many more individual words.

For a function $f:[0,1]\to \mathbb{R}$ and an integer $n\ge 1$, its Bernstein polynomial isThe Bernstein basis supplies nonnegative weights whose sum is one. Equivalently, the weights are the probabilities of a binomial distribution, and the result is an expected value. At a given $x$, this averages the sampled values $f(k/n)$ with weights concentrated near $k/n=x$.^[ref]

The endpoints are exact: $B_{n}f(0)=f(0)$ and $B_{n}f(1)=f(1)$. Interior sampled points need not be interpolated. For example, binomial distribution moments give $B_n(x)=x$ and $B_n(x^2)=x^2+x(1-x)/n$. Thus every affine function is reproduced exactly, whereas $x^2$ is generally changed. Bernstein polynomial degree preservation is a statement about degree of a polynomial, not equality of the entire polynomial.

For a continuous $f$, the uniform continuity of $f$ and the variance $\mathrm{Var}(K/n)=x(1-x)/n\le 1/(4n)$ explain uniform convergence. Split the expected value of the error into $|K/n-x|<\delta$, where uniform continuity makes the change small, and its complement, whose probability is at most $1/(4n{\delta }^2)$ by Chebyshev's inequality. The bounded function $f$ bounds the remaining error. This proves convergence without assuming differentiability.

For $f\in C([0,1{]}^d)$, the tensor-product Bernstein polynomial is the expected value of $f(X_1/n,\dots ,X_d/n)$ for independent random variables $X_j\sim \mathrm{Bin}(n,x_j)$. The same uniform continuity and coordinatewise variance estimates give uniform convergence for each fixed finite $d$.

Mix a binomial distribution with a beta distribution for its success probability: $\Theta \sim \mathrm{Beta}(\alpha ,\beta )$ and $Y\mid \Theta \sim \mathrm{Binomial}(n,\Theta )$. Integrating their densities gives the displayed mass for $y=0,\dots ,n$. Its expectation is $n\alpha /(\alpha +\beta )$, and its variance is $n\alpha \beta (\alpha +\beta +n)/[(\alpha +\beta {)}^2(\alpha +\beta +1)]$. It is the posterior predictive distribution for repeated Bernoulli trials after a beta-prior update.

Take $n$ independent steps, each $+1$ with probability $p>1/2$ and $-1$ with probability $q=1-p>0$. Reverse a step sequence by reading it backward and changing each sign; use the same biased law for both experiments. With $K$ positive steps, the forward sequence weight is $p^K{q}^{n-K}$ and its reversed weight is $p^{n-K}{q}^K$. Their logarithm ratio isHere $S_n$ is dimensionless entropy production, not an entropy in joules per kelvin. A Markov chain on a three-state ring gives a state-space realization: start uniformly, take a clockwise step with probability $p$ and a counterclockwise step with probability $q$. The uniform stationary distribution makes endpoint weights cancel. A physical interpretation as exchanged heat requires local detailed balance with bath entropy $\pm k_{B}a$ per directed step; the abstract model alone specifies path probabilities.

For an allowed value $s=a(2k-n)$, the binomial distribution givesThis is an exact same-law finite fluctuation theorem, with no long-time approximation. For $p=3/4$, $n=9$ and $k=6$, $s=3\ln 3$ and $p_n(s)/p_n(-s)=27$. The positive mean is $⟨S_n⟩=na(p-q)$, although the all-negative path has nonzero probability $q^n$. Directly, $⟨e^{-S_n}⟩=(p{e}^{-a}+q{e}^a{)}^n=(q+p{)}^n=1$.

The left panel shows exact probability masses for $p=3/4$, with negative values highlighted; the right panel tests the logarithm ratio against $s$. These are model values, without experimental measurements or simulated measurement noise.

A Galton-Watson process whose offspring law is a binomial distribution $\mathrm{Bin}(n,p)$ has mean $np$ and the displayed offspring probability generating function. Its branching extinction probability is the smallest root of $q=f(q)$ in $[0,1]$. It dominates a breadth-first exploration of a binomial random graph by allowing each explored vertex up to $n$ independent possible children, including previously exposed or already discovered vertices as phantom children.

The binomial distribution counts successes in a fixed number $n$ of independent Bernoulli trials with common success probability $p$. Equivalently,Here $n$ is a nonnegative integer and $0\le p\le 1$. The binomial coefficient $\left(\genfrac{}{}{0ex}{}{n}{k}\right)=n!/[k!(n-k)!]$ counts placements of the $k$ successes; each particular arrangement has probability $p^k(1-p{)}^{n-k}$. The binomial theorem makes the masses sum to $[p+(1-p){]}^n=1$. At $p=0$ all mass is at zero, and at $p=1$ all mass is at $n$; these endpoint laws can be read directly without interpreting $0^0$.

The linearity of expectation and addition of variances for independent random variables giveThe fixed trial count, common success probability, and joint independence are substantive model assumptions. Sampling uniformly without replacement from a finite population instead leads to a hypergeometric distribution. Independent trials with different success probabilities generally give a Poisson binomial distribution. Common marginal success probabilities alone are insufficient: if all ten indicators equal the same Bernoulli random variable with $p=0.3$, their sum is zero with probability $0.7$ or ten with probability $0.3$. Its mean is still three, but its variance is $100(0.21)=21$. The standard construction is given in Penn State STAT 504, binomial distribution.

For an exact constructed example, let each of ten independent trials choose uniformly from ten labelled tokens, with replacement, and mark three tokens as successes. Then $p=3/10$ and $K\sim \mathrm{Bin}(10,0.3)$. Exactly three successes have probabilityThe mean is $10(0.3)=3$, the variance is $10(0.3)(0.7)=2.1$, and the standard deviation is $\sqrt{2.1}\approx 1.4491$. Zero successes have probability $0.7^{10}=0.0282475249$, so at least one success has probability $1-0.7^{10}=0.9717524751$. A single count calculation replaces enumeration of all $10^{10}$ equally likely token sequences.

Each bar's height is one probability mass. Three is the most likely count, but its mass is only about $26.7\%$: a most likely outcome need not occur most of the time.

For $N$ independent groups each containing $m$ independent Bernoulli trials with common probability $\theta$, the group count has a binomial distribution. If $O_j$ groups have count $j$, the maximum-likelihood estimate is $\hat{\theta }={\sum }_{j}j{O}_j/(mN)$. Form the Pearson chi-squared statistic using $E_j=N\left(\genfrac{}{}{0ex}{}{m}{j}\right){\hat{\theta }}^j(1-\hat{\theta }{)}^{m-j}$. With all $m+1$ categories retained, an interior true parameter, and sufficiently large expected counts, its null limit is a chi-squared distribution with $m-1$ statistical degrees of freedom: one degree is removed by the fixed total and another by fitting $\theta$. Small expected counts require care with this approximation. If categories are pooled, the fitted parameter and calibration should correspond to the pooled statistical model; simply pooling a statistic after fitting the unpooled model does not automatically give the usual chi-squared limit.

For $m$ independent binomial distribution observations with fixed trial count $n$ and total successes $s$, the likelihood-power probability density function relative to $dp$ is $p^{s/T}(1-p{)}^{(mn-s)/T}$. Its normalized law is the displayed beta distribution. Its mean tends to $s/(mn)$ and its variance tends to zero as $T\downarrow 0$, including endpoint maxima. Energy for maximizing a log-likelihood is its negative; the opposite sign reverses the optimization.

The binomial distribution with parameters $n$ and $p$ assigns the displayed probability to $k$ successes, for integers $0\le k\le n$. Each specified sequence of independent trials has probability $p^k(1-p{)}^{n-k}$, and the binomial coefficient counts the sequences with exactly $k$ successes. Outside this range the probability is zero.

A confidence interval for the success probability of a binomial distribution, based on the observed success count and number of trials. A nominal confidence level need not equal the actual finite-sample coverage probability; the distinction matters especially for small samples and proportions near zero or one.

A random variable with a binomial distribution.

Retain each of $N$ objects independently with probability $r$, independently of the nonnegative integer-valued random variable $N$. The retained count $Y$ has the displayed conditional binomial distribution. The probability generating function of an independent random sum gives $G_Y(s)=G_N(1-r+rs)$. If $N$ has finite variance, the law of total expectation and law of total variance giveThe first term is the average conditional variance of the independent retention decisions; the second is the variance in their conditional expected value.