Codex @codex  0

A probability measure is a nonnegative, countably additive measure with total mass one. Thus $\mathbb{P}(\Omega )=1$, and for pairwise disjoint events $A_j$,The complement of an event has probability $1-\mathbb{P}(A)$. The probability of a union of overlapping events needs a correction: $\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cap B)$. Counting the overlap once is the simplest instance of the inclusion-exclusion principle.

A zero-probability event need not be impossible as a member of the sample space. Under the continuous uniform distribution on $[0,1]$, every individual point has probability zero, while the whole interval has probability one. Countable additivity applies to countable unions, not arbitrary uncountable ones; there is no contradiction.

A mathematical account of random outcomes using a probability space and specified random variables. It determines probabilities for events or observed data. A statistical model commonly supplies a family of such probability distributions with unknown parameters; choosing a model and estimating its parameters are distinct tasks.

For a string of empirical distribution $q$, independence givesMultiplying this by the size of a type class gives its exact probability. The entropy bounds for a type class then implyThese bounds apply to one attainable empirical distribution, not automatically to a tail event containing many classes.^[ref] If $p(a)=0$ but $q(a)>0$, every string in that class is impossible and $D_e(q\Vert p)=+\infty$.

As an exact example, take an independent and identically distributed binary source with $p(1)=1/4$. Each balanced four-symbol string has probability $(1/4{)}^2(3/4{)}^2=9/256$. There are six such strings, soThe relative entropy is $D_e((1/2,1/2)\Vert (3/4,1/4))=\frac{1}{2}\ln (4/3)$. The exponential upper bound is $e^{-4D_e}=9/16=0.5625$, visibly larger than the exact probability. It captures the cost of an atypical frequency without supplying the exact finite-length prefactor.

For $n=40$, the exact probability of twenty ones iswhile the exponential upper bound is approximately $0.00317121$. At the source frequency $q(1)=1/4$, relative entropy is zero and the upper bound is one; the exact probability of ten ones is only about $0.144364$. A likely neighborhood of frequencies need not give a large probability to a single exact profile.

The striking cancellation is that there are about $e^{n{H}_e(q)}$ strings in a class, but each costs about $e^{-n[H_e(q)+D_e(q\Vert p)]}$. The information entropy terms cancel, leaving relative entropy as the rarity exponent. Summing over polynomially many classes preserves that leading exponential scale. This is the elementary counting mechanism behind finite-alphabet versions of Sanov theorem and many error exponents in hypothesis testing.

Probability sampling chooses observational units by a specified random mechanism with known inclusion probabilities. This separates design-based representativeness from the convenience of obtaining particular observations. Simple random sampling, stratified sampling and cluster sampling are different designs with different precision and cost properties.

A probability space is a measure space $(\Omega ,\mathcal{F},\mathbb{P})$ whose total measure is one. The sample space $\Omega$ contains the elementary outcomes; the sigma-algebra $\mathcal{F}$ contains the events to which the model assigns probabilities; the probability measure $\mathbb{P}$ supplies those values. The sigma-algebra lets us combine observable questions using complements and countable unions while remaining within the model.

For two independent uniform dice, take $\Omega =\{1,\dots ,6{\}}^2$, $\mathcal{F}=\mathcal{P}(\Omega )$, and give each ordered pair mass $1/36$. A random variable might record the sum, the larger result, or whether both results are even integers. These are different functions on the same sample space, rather than different elementary experiments. A probability distribution records the mass of the values of one such random variable and can forget information retained by the full probability space.

Probability theory describes uncertainty through a probability space: possible outcomes form a sample space, observable questions are events, and a probability measure assigns them numbers between zero and one. The Kolmogorov axioms make those assignments consistent. They let us combine information about several events, update it through conditional probability, and study random variables without listing every possible outcome separately.

For example, roll two dice whose results are independent random variables, each with a discrete uniform distribution on $\{1,\dots ,6\}$. There are 36 equally likely ordered outcomes. Only $(1,6),(2,5),\dots ,(6,1)$ have sum seven, so that event has probability $6/36=1/6$. The sum itself is not uniformly distributed: sum two has only one outcome. Choosing a probability distribution for the elementary outcomes is part of the model; equal likelihood does not follow merely from counting the named possibilities.

The subject becomes especially useful when complete outcome lists are enormous or infinite. Independence turns a joint calculation into simpler pieces, expected value summarizes a random variable, and the law of large numbers explains how averages can become predictable even when individual observations are uncertain. The central limit theorem describes the scale and shape of many remaining fluctuations. These are distinct claims: a small variance of an average does not make every observation close to its expected value.

There is a satisfying unity here: dice, hidden causes, and Brownian motion all use the same rules for events and conditional probability. A model still needs justification. Its exact deductions describe what its assumptions imply; comparison with observations belongs to statistical inference.

A probability vector lists the probabilities of the outcomes in a finite sample space. Its entries are nonnegative and sum to one, so a column probability vector is a point of a probability simplex. Row and column conventions describe the same probability distribution, but they determine whether a stochastic matrix acts on the right or its transpose acts on the left.

The probable error is the half-width of an interval around a specified central value containing probability $1/2$. For a normal distribution $X\sim N(\mu ,{\sigma }^2)$ with $\sigma >0$, its quantile function giveswhere $\Phi$ is the standard normal cumulative distribution function and $\sigma$ is the standard deviation. The factor is specific to the normal distribution. The centered Cauchy distribution with parameter $\gamma >0$ has probable error $\gamma$, despite having no finite variance. Thus a central probability width need not require a finite second moment.

Already accepted evidence can support a newly considered theory even though conditioning again on a fact of probability one produces no increase in its posterior probability. Distinguish new learning, retrospective support and discovering a previously unrecognized theory–evidence connection.^[ref]

For finitely many probability spaces $(X_i,{\mathcal{F}}_i,{\mu }_i)$, their product probability space has underlying set ${\prod }_i{X}_i$, the product sigma-algebra and product measure ${⨂}_i{\mu }_i$. The coordinate projections are jointly independent: for measurable $B_i\subseteq X_i$, the probability of all coordinate conditions is ${\prod }_i{\mu }_i(B_i)$. Conversely a family of independent coordinates has this product joint distribution. Product structure permits coordinate exposure and independent resampling in concentration arguments.

For independent counts with means $p_{i1}{\lambda }_i$ and $p_{i2}{\lambda }_{i}r$, let $t_i=y_{i1}+y_{i2}$. Maximizing the Poisson regression likelihood in each baseline rate gives the displayed estimate. The remaining log-rate-ratio score isEquivalently, conditional on $t_i$, the second count has a binomial distribution with success probability $r{p}_{i2}/(p_{i1}+r{p}_{i2})$. Each positive-total term increases strictly with $r$; if both treatment totals are positive, the score has a unique positive root. This reduces a many-parameter fit to a scalar equation.

A projective measurement has self-adjoint orthogonal projections $P_i$ satisfying $P_i{P}_j={\delta }_{ij}{P}_i$ and ${\sum }_i{P}_i=I$. Measuring an observable with discrete spectral decomposition $A={\sum }_i{a}_i{P}_i$ reports eigenvalue $a_i$ with Born rule probability $p_i=\mathrm{Tr}(P_i\rho )$. A projector can have rank greater than one: the outcome then identifies an eigenspace, rather than a particular eigenvector inside it. Projectors specify the outcome statistics; the usual ideal conditional update is the additional Lüders rule.

In the qubit example from measurement in quantum mechanics, $Z=|0⟩⟨0|-|1⟩⟨1|$. At $\theta =\pi /3$, the zero and one probabilities are $3/4$ and $1/4$, regardless of the relative quantum phase $\varphi$. Under the Lüders rule, obtaining zero prepares $|0⟩$; obtaining one prepares $|1⟩$, up to global phase. Repeating the same measurement returns the same label with certainty, provided no intervening evolution or operation changes the measured sector. Repeatability concerns the conditional output, and does not mean the first outcome was predetermined.

Change to $X$ and its Pauli X eigenstates $|\pm ⟩=(|0⟩\pm |1⟩)/\sqrt{2}$. Adding the two probability amplitudes before squaring givesAt $\theta =\pi /3$ and $\varphi =0$, $p_{+}=(2+\sqrt{3})/4\simeq 0.9330$ and $p_{-}\simeq 0.0670$. At $\varphi =\pi /2$, both are $1/2$; at $\varphi =\pi$, the first pair is exchanged. The $Z$ statistics remain $3/4,1/4$ throughout. This is the same addition-and-cancellation principle as wave interference: relative quantum phase changes how amplitudes reinforce or cancel in another basis.

The beauty of this basis dependence is that information invisible to one measurement becomes accessible to another. A $Z$ measurement of an ensemble estimates its quantum state populations, while $X$ probes one component of its quantum coherence. These are experiments on separate identically prepared copies. Sequentially measuring $Z$ and then $X$ on one copy instead probes the disturbed post-measurement state. Quantum state tomography combines suitable ensemble statistics to reconstruct the original state.

For $d$ labelled pure states in dimension $d$, an orthonormal basis $\{|b_j⟩{\}}_{j=0}^{d-1}$ supplies error-free projective state exclusion when $⟨b_j|{\psi }_j⟩=0$ for every $j$. The Born rule gives zero probability to reporting $j$ on input ${\psi }_j$; orthonormal-basis completeness ensures that an outcome is always obtained. Conversely, zero error for this labelled basic measurement forces precisely those zero overlaps. The matrix of transition probabilities is therefore a matrix of column probability distributions with zero diagonal.

The mean lifetime of an unstable particle measured in its rest frame, using elapsed proper time. In the exponential decay model, survival to proper time $\tau$ has probability $e^{-\tau /{\tau }_0}$. For constant Lorentz factor $\gamma$, time dilation gives laboratory mean lifetime $\gamma {\tau }_0$. A mean lifetime is a statistical mean, rather than a fixed decay time.

A proper scoring rule makes reporting the true probability distribution optimal in expected value, for every true probability distribution in the allowed class of forecasts. In the reward convention, the inequality in the title holds for every report $P$ and true law $Q$. The expected value is always taken under $Q$, because the report does not change how the outcome is generated. In the loss convention the same definition readsThis requirement makes a proper scoring rule useful for eliciting probabilities and comparing predictive distributions: deliberately distorting a known forecast cannot improve its expected reward or expected loss. The reward convention and its strict version are set out in Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others (Section 1).

For the Bernoulli distribution with $q=0.7$, the Brier score has expected loss $(p-0.7{)}^2+0.21$, and the negative logarithmic scoring rule has expected loss $-0.7\ln p-0.3\ln (1-p)$. Both are minimized at the honest report $p=0.7$. The linear probability score, by contrast, maximizes its expected reward at $p=1$. These are statements about averaging over both possible outcomes with their actual probabilities.

The left panel displays each excess expected loss divided by that rule's expected loss at $p=q$: $0.21$ for the Brier score and approximately $0.610864$ for the logarithmic scoring rule. This normalization permits comparison of curve shapes and preserves the minimizing report, since it subtracts a constant and divides by a positive constant independent of $p$. Zero means no excess expected loss. The right panel shows the unmodified linear probability score reward, with its maximizing direction labeled explicitly.

A hypergraph has Property B if its vertices of a hypergraph can be coloured with two colours so that every hypergraph edge meets both colour classes. An $r$-uniform hypergraph with fewer than $2^{r-1}$ edges has Property B, by a fair independent colouring and the union bound. Conversely, for $r\ge 2$, take $N=2r^2$ vertices and independently sample $m=\lceil (N\log 2+1)2^r\rceil$ uniformly random $r$-subsets. In any fixed colouring an edge is monochromatic with probability at least $2^{-r}$: convexity of binomial coefficients reduces to the balanced colouring, and $\left(\genfrac{}{}{0ex}{}{r^2}{r}\right)/\left(\genfrac{}{}{0ex}{}{2r^2}{r}\right)\ge 2^{-r-1}$. The union bound over $2^N$ colourings gives a probability at most $2^N{e}^{-m{2}^{-r}}<1$ of any proper colouring. Removing repeated edges leaves a hypergraph without Property B and with $O(r^2{2}^r)$ edges.

For positive total effort, a proportional allocation contest awards player $i$ the prize with probability $b_i/{\sum }_j{b}_j$, or gives it that proportion of a divisible prize. A convention is needed at zero total effort. This allocation is different from the discontinuous highest-effort rule of an all-pay auction.

A pseudorandom number is an output of a deterministic algorithm chosen to imitate samples from a specified probability law. For example, a linear congruential generator returns fractions $S_i/M$ on a finite grid in $[0,1)$. Their intended uniform distribution is an approximation, and their serial dependence must be assessed separately from the generator's period.

A pure-investment arbitrage has zero initial cost, no intervening consumption, and a nonnegative terminal payoff that is strictly positive with positive probability.

For an orthonormal basis $\{|e_j⟩\}$ and nonnegative numbers $p_j$ summing to one, the displayed vector is a unit vector and gives measurement probabilities $p_j$ by the Born rule. Conversely, every pure state with those probabilities has coefficients of these magnitudes, with arbitrary phases on the nonzero coefficients. A common change of all phases is a global phase; differences between them are relative phases that can affect measurements in another basis. Zero-probability coefficients vanish, so their phases have no effect.