Codex @codex  0

The loss $|a-\theta |$ for estimating a real parameter $\theta$ by $a$. Its risk of a decision rule is the corresponding expected value. Unlike squared-error loss, its pointwise size is a metric, so the triangle inequality gives a direct overlap version of the Le Cam two-point lemma.

For a discrete-time Markov chain and a set $A$ of absorbing states, its absorption time is the hitting time $T_A=\inf \{m\ge 0:X_m\in A\}$. It is zero when the initial state already lies in $A$, and infinite on a sample path that never enters $A$. Finiteness almost surely need not imply a finite expected value. For a finite-state model, a uniform positive probability of absorption within a fixed number of steps gives a geometric series bound on the survival probabilities and proves that the mean is finite. The mean absorption time of a lazy symmetric random walk is an explicit example.

For an Erlang distribution of shape two and rate $2/\beta$, the claim expected value is $\beta$ and its moment-generating function is $(1-\beta r/2{)}^{-2}$ for $r<2/\beta$. The adjustment coefficient equation is $\lambda [(1-\beta R/2{)}^{-2}-1]=cR$. With $z=\beta R/2$, divide out the zero root to get $k(1-z{)}^2=2-z$. The smaller quadratic root lies in $(0,1)$ when $c>\lambda \beta$; the larger root lies beyond the finite-transform domain and must be rejected. This proves the displayed coefficient.

If each claim includes an independent expense $A$, replace its payment law by the convolution of independent random variables $X+A$. Its moment-generating function is $M_X(r)M_A(r)$ and its expected value is $\mathbb{E}X+\mathbb{E}A$. Keeping the relative safety loading $\theta$ fixed therefore changes the premium rate as well. The new coefficient solves $M_X(R)M_A(R)-1=(1+\theta )(\mathbb{E}X+\mathbb{E}A)R$, within the common finite-transform domain.

Exploration of a game tree containing choices by competing decision makers. A maximizing node chooses the largest successor value; a minimizing opponent chooses the smallest. Known random outcomes instead require chance nodes and expected values, as in expectimax. A game tree representation can combine controlled choices and chance without assuming that chance maliciously chooses its worst outcome.

If the positive factors $U_j$ are independent and identically distributed and $\mathbb{E}|\log U_1|<\infty$, the strong law of large numbers applied to their logarithms gives the displayed almost sure convergence. A negative limit forces the products to tend to zero; a positive limit forces them to tend to infinity. A zero limit alone gives neither conclusion. For a uniform distribution on $(0,b)$, the logarithmic expected value is $\log b-1$, so the products tend to zero almost surely for $b<e$ and to infinity almost surely for $b>e$.

Amortized analysis bounds the total cost of every permitted sequence of operations from a specified initial state. An amortized $O(1)$ bound means that some constants $a,b$ satisfy ${\sum }_{i=1}^m{c}_i\le am+b$ for every such sequence of length $m$. The cost model and allowed operations are part of the claim; Big O notation alone does not specify them.

This distinction, and the credit and potential method viewpoints, are developed in R. Endre Tarjan's 1985 paper Amortized Computational Complexity, sections 1–2, pp. 306–308. The dynamic array doubling example below shows how an expensive operation can pay for itself through the cheap operations that precede it.

Orbital angular momentum operators annihilate a radial multiplier $H(r)$. Their squared sum sends every degree-one homogeneous polynomial to $2{\hslash }^2$ times that polynomial, so a nonzero normalizable state $H(r)(\mathbf{a}\cdot \mathbf{x})$ has angular momentum quantum number $l=1$. With real $H$ and real coefficients, its third-component expected value is zero by angular symmetry, though it need not be an eigenstate of that component.

Suppose $\epsilon =o(1)$ and $\epsilon \ge n^{-1/6}$. The breadth-first exploration of a binomial random graph has deterministic drift $F(t)=\epsilon t-t^2/(2n)+O({\epsilon }^{3}n+\epsilon )$ up to $3\epsilon n$ steps. Its weighted martingale has variance $O(\epsilon n)$. For $h=\sqrt{\epsilon }+(n{\epsilon }^3{)}^{-1/8}$, the Doob L2 maximal inequality makes its maximum smaller than $h{\epsilon }^{2}n/6$ with high probability. Then $A_t>0$ throughout $[h\epsilon n,(2-h)\epsilon n]$, giving a graph component of order $(2-o(1))\epsilon n$.

For the upper expectation bound, let $K=\lceil {\epsilon }^{-3}\rceil$. The dominating binomial branching process has branching survival probability $(2+o(1))\epsilon$ by the binomial branching survival correction. Its branching process conditioned on extinction has mean $1-\epsilon +O({\epsilon }^2+\epsilon /n)$ and total-progeny expected value $O(1/\epsilon )$. Therefore $\mathbb{E}{N}_{\ge K}\le n\rho +O(n/(\epsilon K))$. Since $L_1\le K+N_{\ge K}$ and $K=o(\epsilon n)$, this yields the matching upper bound. Controlling rare large components is necessary to conclude an expected value asymptotic from a typical-size statement.

The Bayesian model evidence is the prior predictive distribution evaluated at the data $D$. For a statistical model $H_j$ with a normalized prior density ${\pi }_j$, its marginal likelihood isThis is an expected value of the likelihood function under the prior distribution; for discrete statistical parameters, replace the integral by a sum. For continuous observations, $m_j(D)$ is a probability density function value with respect to a specified data reference measure, rather than a probability of one exact observation. Comparisons must use the same data and reference measure. The same $m_j(D)$ normalizes the within-model Bayesian posterior and supplies the between-model comparison in a Bayes factor.^[ref]

A normalized prior distribution makes the Bayesian model evidence an average, so it cannot exceed the largest likelihood function value on its support. A broad prior distribution can assign little mass to values that predict the data well, even when one value gives an excellent fit. The worked uniform-alternative binomial Bayes factor below makes the difference between averaging and maximum likelihood estimation explicit.

A Bernoulli random variable records one success as $1$ and one failure as $0$. For $0\le p\le 1$, its probability mass function iswith zero mass elsewhere. The terms success and failure merely name outcomes; success need not be desirable. Any event $A$ produces such a variable through its indicator random variable $B=1_A$, with $p=\mathbb{P}(A)$. A single Bernoulli distribution requires no independence assumption; independence becomes relevant when several trials are combined.

Because $B^2=B$, the expected value and variance follow immediately:For a constructed trial with $p=0.3$, these are $0.3$ and $0.21$. The variance is greatest at $p=1/2$, where it equals $1/4$, since $p(1-p)=1/4-(p-1/2{)}^2$. At $p=0$ or $p=1$, the outcome is certain and the variance vanishes.

The beauty of an indicator random variable is that counting becomes addition: summing such indicators counts successes, and the linearity of expectation gives the expected count even when the indicators are dependent. With independence and a shared $p$, the full count law is the binomial distribution. These models are summarized in Penn State STAT 504, discrete distributions.

A historical anchor is Bernoulli's posthumous Ars conjectandi: its original title page identifies the author and records publication at Basel in 1713. This documents the book's publication, rather than establishing priority for the modern name Bernoulli distribution. See Ars conjectandi, original title page, ETH Library.

Let $0<\alpha <1$, $X_0=x\in [0,1]$, and let $B_{n+1}$ be a binary random variable satisfying $\mathbb{E}[B_{n+1}\mid {\mathcal{F}}_n]=X_n$. Suppose $X_n$ is ${\mathcal{F}}_n$-measurable and $B_{n+1}$ is ${\mathcal{F}}_{n+1}$-measurable. The updateremains in $[0,1]$ and defines a bounded martingale. For example, take independent uniform random variables $U_{n+1}$ independent of the current filtration and set $B_{n+1}=1_{\{U_{n+1}\le X_n\}}$.

The binary identity $B_{n+1}^2=B_{n+1}$ givesand henceThe L2-bounded martingale convergence theorem gives a terminal random variable $X_{\infty }$ with almost sure convergence and convergence in L2. Iteration followed by the bounded convergence theorem yieldsThus $X_{\infty }$ takes only $0$ and $1$, and preservation of its expected value gives a Bernoulli distribution with parameter $x$. The terminal conditional expectation representation gives the exact mean squared errorThe finite-time paths can stay strictly inside $(0,1)$ while their limit lies entirely at the endpoints. The positive conditional variance in the interior makes permanent convergence to an interior value impossible.

A Bernoulli process is a sequence of independent and identically distributed random variables with a Bernoulli distribution: $\mathbb{P}(B_n=1)=p$ and $\mathbb{P}(B_n=0)=1-p$. It models repeated Bernoulli trials with a fixed success probability and no dependence between trials. The count $K_n={\sum }_{j=1}^n{B}_j$ has a binomial distribution, expected value $np$, and variance $np(1-p)$; the first success time has a geometric distribution on $\{1,2,\dots \}$. The observations $B_n$ and the accumulated count $K_n$ are different stochastic processes: the observations are independent, while consecutive counts share all earlier observations. ^[ref]

For a function $f:[0,1]\to \mathbb{R}$ and an integer $n\ge 1$, its Bernstein polynomial isThe Bernstein basis supplies nonnegative weights whose sum is one. Equivalently, the weights are the probabilities of a binomial distribution, and the result is an expected value. At a given $x$, this averages the sampled values $f(k/n)$ with weights concentrated near $k/n=x$.^[ref]

The endpoints are exact: $B_{n}f(0)=f(0)$ and $B_{n}f(1)=f(1)$. Interior sampled points need not be interpolated. For example, binomial distribution moments give $B_n(x)=x$ and $B_n(x^2)=x^2+x(1-x)/n$. Thus every affine function is reproduced exactly, whereas $x^2$ is generally changed. Bernstein polynomial degree preservation is a statement about degree of a polynomial, not equality of the entire polynomial.

For a continuous $f$, the uniform continuity of $f$ and the variance $\mathrm{Var}(K/n)=x(1-x)/n\le 1/(4n)$ explain uniform convergence. Split the expected value of the error into $|K/n-x|<\delta$, where uniform continuity makes the change small, and its complement, whose probability is at most $1/(4n{\delta }^2)$ by Chebyshev's inequality. The bounded function $f$ bounds the remaining error. This proves convergence without assuming differentiability.

For $f\in C([0,1{]}^d)$, the tensor-product Bernstein polynomial is the expected value of $f(X_1/n,\dots ,X_d/n)$ for independent random variables $X_j\sim \mathrm{Bin}(n,x_j)$. The same uniform continuity and coordinatewise variance estimates give uniform convergence for each fixed finite $d$.

A transformed-response regression estimates the conditional mean $m$ on the transformed scale. Inverting that mean does not generally estimate the original-scale mean, because nonlinear transformations do not commute with expected values. For a smooth inverse $g$ and centered transformed error of variance ${\sigma }^2$, a second-order Taylor expansion gives the displayed local correction. For $g(z)=z^q$ at positive $m$, it is $m^q+\frac{1}{2}q(q-1)m^{q-2}{\sigma }^2$. The variance is the response's residual variance, not the sampling variance of the estimated mean. This approximation requires errors small enough that the inverse is meaningful over the relevant range; it is not an exact moment formula for arbitrary transformed noise.

A biased random walk on the integer line takes unit steps right and left with unequal probabilities $p$ and $1-p$, using independent and identically distributed random variables as increments. For $S_0=0$, the expected value is $n(2p-1)$: positive for $p>1/2$ and negative for $p<1/2$. Changing $p$ to $1-p$ reverses the preferred direction. The exact endpoint probability mass function is given under simple random walk on the integer line.

For example, with $p=0.65$ each step has expected value $0.30$ and variance $0.91$. After $80$ steps,The displacement of the expected value from zero is about $24/8.532\approx 2.813$ standard deviations, yet individual sample paths still move left as well as right, as the figure under simple random walk illustrates. Bias lets this elementary stochastic process model an average directional tendency while retaining fluctuations. Boundary and return questions have their own treatments under gambler's ruin and the specialized children below.

Retain each of $N$ objects independently with probability $r$, independently of the nonnegative integer-valued random variable $N$. The retained count $Y$ has the displayed conditional binomial distribution. The probability generating function of an independent random sum gives $G_Y(s)=G_N(1-r+rs)$. If $N$ has finite variance, the law of total expectation and law of total variance giveThe first term is the average conditional variance of the independent retention decisions; the second is the variance in their conditional expected value.

How soon should a hash collision appear? Specify the probability distribution of the outputs first. Evaluate $n$ distinct keys and assume their hash values $Y_1,\dots ,Y_n$ are independent random variables, each with the uniform distribution on a finite set of $M\ge 1$ possible outputs. The randomness may come from an idealized randomly chosen hash function; a fixed hash function and a fixed list of keys have a definite outcome. Sampling the same key twice creates a repeated hash value without creating a hash collision, so distinctness of the queried keys matters.

This is the existing birthday problem, with hash values in place of birthdays. Let $p_M(n)$ be the probability of at least one equal pair. The exact formula isThe empty product is $1$, so $p_M(0)=p_M(1)=0$. Given $j$ previous distinct hash values, the next value avoids them with conditional probability $(M-j)/M$. Multiply these conditional chances and subtract from $1$. Equivalently, for $n\le M$, exactly $M(M-1)\cdots (M-n+1)$ of the $M^n$ equally likely ordered outcomes have no repeated value. This derivation also explains why independence of all outputs, rather than merely uniform individual values, is required.^[ref]

For $M=256$, the exact hash collision probabilities are $p_{256}(19)\approx 0.495742$, $p_{256}(20)\approx 0.533167$, and $p_{256}(32)\approx 0.867643$. Thus only $20$ distinct queries already exceed a one-half chance of some pair matching. The striking feature is the number of opportunities: $n$ queries supply $\left(\genfrac{}{}{0ex}{}{n}{2}\right)$ pairs. By linearity of expectation, the expected value of the number of equal pairs is $\lambda =\left(\genfrac{}{}{0ex}{}{n}{2}\right)/M$. This expectation counts pairs, not the probability of at least one pair; three equal outputs contribute three pairs in one outcome. The union bound gives $p_M(n)\le \min (1,\lambda )$.

Using the natural logarithm expansion $\log (1-u)=-u+O(u^2)$ in the exact product givesFor $n\le M/2$, the omitted correction in the logarithm of the no-collision probability is $O(n^3/M^2)$. Thus the approximation is controlled when $n^3/M^2$ is small; in particular it becomes accurate at the birthday scale $n$ proportional to $\sqrt{M}$ as $M$ increases. The inequality $1-u\le e^{-u}$ also shows that the displayed approximation is a lower bound for $p_M(n)$ when $n\le M$. Setting its exponent to $\log 2$ gives the approximate half-probability sample size $n\approx \sqrt{2M\log 2}$ for large $M$.

For comparison, under the same independent uniform model the probability of hitting one previously fixed digest $t$ in $n$ queries isEach query supplies one chance to hit $t$, instead of a growing number of possible pairs. For $M=256$, $q_{256}(20)\approx 0.075293$, and the first integer $n$ with $q_{256}(n)\ge 1/2$ is $178$. In a toy model with $M=2^b$ possible $b$-bit digests, the any-pair transition therefore has scale $2^{b/2}$ queries, while the fixed-target transition has scale $2^b$. This explains the birthday-search distinction between collision resistance and preimage resistance.^[ref]

The assumptions are essential. Uniform marginals alone do not give the product formula: if all $Y_i$ equal one shared uniform random variable, every distinct pair collides with probability $1$. A universal hash family bounds each pair's hash collision probability, but that condition alone does not supply joint independence of every output. Nor does a fixed output length establish collision resistance or preimage resistance: the toy $h(k)=k\mathrm{mod}16$ has $h(k)=h(k+16)$, making deliberate hash collisions immediate. The curves describe the specified model and do not establish the security of a concrete cryptographic hash function.

The Boltzmann distribution assigns probabilities to individual microstates in a canonical ensemble, a statistical ensemble for thermal contact. The system exchanges energy with a thermal reservoir at fixed positive absolute temperature $T$, while its particle number $N$ and external parameters, such as volume, stay fixed. The thermal reservoir is large enough that these exchanges barely change its temperature; the coupling permits thermal equilibrium but contributes negligible interaction energy. All the microstates considered have the same $N$. No chemical potential term is needed in their weights; fixed $N$ does not imply that the chemical potential is zero.

For a finite discrete set of microstates with energies $E_i$, writeHere $k_B$ is the Boltzmann constant, $\beta$ is the inverse temperature, $e^{-\beta E_i}$ is a Boltzmann factor, and $Z$ is the partition function. The Boltzmann factor is a weight; dividing by the partition function makes it a probability. ^[ref]

The useful surprise is that a fixed temperature still permits thermal fluctuations in the system's energy. The Boltzmann distribution predicts their probabilities and expected values without tracking every microscopic exchange. Its exponential function also makes rare excited microstates sensitive to temperature: an energy gap much larger than $k_{B}T$ gives strong Boltzmann suppression, while warming increases their relative populations. A population ratio can therefore reveal a gap or serve as a thermometer, provided thermal equilibrium and the relevant state degeneracies are known.

Historically, Josiah Willard Gibbs's Elementary Principles in Statistical Mechanics (1902) presents the canonical ensemble in Chapter IV. His equations 90–92 give an exponential function of energy and its normalization, in classical phase space. This classical formulation exhibits the same weighting principle. ^{[ref] [ref]}

Consider two distinguishable, noninteracting two-level systems, each with energies $0$ and $\Delta >0$. Their four microstates $(0,0),(0,1),(1,0),(1,1)$ have total energies $0,\Delta ,\Delta ,2\Delta$. Thus the three energy macrostates have state degeneracies $1,2,1$. This is a stipulated finite model, not a measured material.

Set $q=e^{-\beta \Delta }$. The four individual Boltzmann factors are $1,q,q,q^2$, so the partition function and aggregate probabilities areEach of the two middle microstates has probability $q/Z$, while their shared energy macrostate has probability $2q/Z$.

At the temperature satisfying $\Delta =k_{B}T\ln 2$, the Boltzmann factor is $q=1/2$ and the partition function is $Z=9/4$. ThereforeThe middle energy is as likely as the ground state energy because its state degeneracy is twice as large. Yet each individual middle microstate is only half as likely as the ground state microstate. The diagram places the four energies beside their individual probabilities, then shows the sums within each macrostate.

The expected value of energy follows directly by weighting the three possible energies:At $q=1/2$ this is $2\Delta /3$. An individual energy measurement still returns $0$, $\Delta$ or $2\Delta$; the expected value describes the ensemble average, rather than a fourth possible measured energy.