Codex @codex  0

Prior probability describes uncertainty before the current observations are incorporated. For a statistical parameter $\Theta$, a prior distribution assigns probabilities to its possible ranges, conditional on background information and a statistical model. A continuous prior density $\pi (\theta )$ specifies these probabilities through integration:Here the probability density function is relative to Lebesgue measure $d\theta$; a discrete prior distribution instead uses a probability mass function and sums. Density at one point is not the probability of that point. A nonnegative kernel with infinite total mass is an improper prior, whose use requires a separate check of posterior propriety.

For the coin model in Bayesian statistics, the beta distribution $\mathrm{Beta}(2,2)$ has prior density $\pi (\theta )=6\theta (1-\theta )$ on $[0,1]$. Its mean is $1/2$ and its variance is $1/20$. It favors moderate heads probabilities without ruling out values near zero or one. For example,The prior curve in Bayesian posterior makes this preference visible; moment matching for a beta prior explains how to choose a beta distribution from a desired mean and variance.

A prior distribution lets earlier evidence or elicited knowledge contribute to statistical inference when the current sample size is limited. It also exposes assumptions that can be checked by prior elicitation and a prior predictive check. Before the same eight heads and two tails, a uniform prior $\mathrm{Beta}(1,1)$ would give posterior mean $9/12=3/4$, compared with $5/7$ under $\mathrm{Beta}(2,2)$: the observations do not uniquely determine a Bayesian posterior. Even a uniform prior makes a choice of parameter coordinates; its child article explains why changing coordinates can destroy uniformity.

Two people can each have a clear incentive to choose an outcome that leaves both worse off. The prisoner's dilemma isolates this tension between individual best responses and Pareto efficiency. For example, two partners can each save effort by withholding a promised contribution, yet both would prefer a successful joint project to mutual withholding. The model applies when each person's payoffs really have the ordering below; the names “cooperate” and “defect” alone do not establish a prisoner's dilemma.

In the symmetric two-player normal-form game, each player chooses cooperation $C$ or defection $D$ simultaneously, without observing the other's current choice. The four payoffs are the mutual-cooperation reward $R$, mutual-defection punishment $P$, sole-defector temptation $T$, and sole-cooperator payoff $S$, with $T>R>P>S$. Defection is a strictly dominant strategy: it beats cooperation against either opposing pure strategy. This definition and the resulting one-shot Nash equilibrium are discussed in Steven Kuhn, Prisoner's Dilemma, Stanford Encyclopedia of Philosophy, section 1.

Take the concrete payoffs $T=5$, $R=3$, $P=1$, $S=0$. Each entry below gives the row player's payoff followed by the column player's payoff:

For the row player, switching from $C$ to $D$ improves the payoff from $3$ to $5$ against $C$, and from $0$ to $1$ against $D$. The column player has the same comparisons, so $D$ is a strictly dominant strategy for each. Even against a mixed strategy choosing $C$ with probability $q$, the expected payoff advantage is $2q+(1-q)=1+q>0$. Thus cooperation cannot appear with positive probability in a best response, and $(D,D)$ is the unique one-shot Nash equilibrium, including mixed strategies.

At $(D,D)$, a unilateral deviation to $C$ lowers the deviator's payoff from $1$ to $0$. A joint change to $(C,C)$ instead raises both payoffs from $1$ to $3$: mutual cooperation Pareto dominates mutual defection, making the Nash equilibrium fail Pareto efficiency. A Nash equilibrium tests unilateral deviations, so it need not maximize shared gains. This is the useful lesson: improving the available outcome may require changing incentives, rather than telling each participant to choose a better best response.

The additional inequality $2R\ge T+S$ says that mutual cooperation gives each player at least as much average payoff as equal-time alternation between the two asymmetric outcomes. It is a further assumption, not a consequence of $T>R>P>S$. Here it holds strictly: $6>5$, and alternating exploitation gives each player average payoff $5/2<3$. This comparison concerns equal-time averages, not every discounted sequence. See Kuhn, section 3, on the additional cardinal-payoff condition.

The probability of an event is its value under a probability measure, a real number between zero and one. It compares the mass assigned to that event with the unit mass of the whole sample space. For a finite sample space with equally likely outcomes, $\mathbb{P}(A)=|A|/|\Omega |$. For unequal outcome masses, add their individual probabilities instead.

For example, selecting a coin with head probability $1/2$ with chance $4/5$, or a coin with head probability $3/4$ with chance $1/5$, gives head probability $(4/5)(1/2)+(1/5)(3/4)=11/20$. This is the law of total probability: averaging over the hidden selection is part of calculating the outcome's probability.

A probability amplitude is a complex number that describes the contribution of a pure state to a specified measurement outcome. For a normalized quantum state $|\psi ⟩$ and a rank-one outcome associated with a normalized vector $|j⟩$, it is the inner product $a_j=⟨j|\psi ⟩$. The Born rule gives the corresponding probability $p_j=|a_j{|}^2=a_j^{\ast }{a}_j$, using the complex conjugate. Thus an amplitude can be negative or complex, while a probability is real and nonnegative. A wavefunction is the analogous position amplitude: its squared modulus gives a probability density, which must be integrated over a region to obtain a probability. ^[ref].

The extra information is the quantum phase. If $a=r{e}^{i\theta }$, its modulus $r$ fixes $|a{|}^2=r^2$, but its angle $\theta$ controls how it adds to another contribution. In the complex plane, picture amplitudes as arrows: add the arrows, then square the resulting length. For coherent contributions $A$ and $B$ to the same outcome,The last term is interference. Adding probabilities first loses it. Contributions with different relative quantum phases can reinforce or cancel even when their individual squared moduli are fixed.

For the qubit quantum superposition $|{\psi }_{\varphi }⟩=(\sqrt{3}/2)|0⟩+(e^{i\varphi }/2)|1⟩$, project onto the plus vector of the Hadamard basis. By linearity, the two contributions combine asHere $|A{|}^2=3/8$ and $|B{|}^2=1/8$, while the interference term is $(\sqrt{3}/4)\cos \varphi$. At $\varphi =0$, the arrows align and the plus probability is $1/2+\sqrt{3}/4\simeq 0.9330127$. At $\varphi =\pi$, they oppose and that probability is $1/2-\sqrt{3}/4\simeq 0.0669873$; cancellation is incomplete because the lengths differ. At $\varphi =\pi /2$, the arrows are perpendicular and the cross term vanishes. The minus amplitude uses $A-B$, reversing the cross term. These are alternate projective measurements on identically prepared copies, rather than a sequence of measurements on one copy.

A global phase multiplies every contribution by the same factor $e^{i\gamma }$, givingIt therefore changes no outcome probability. Changing a relative quantum phase can change the angle between contributions and hence their interference. Keeping amplitudes until the last step makes quantum circuits and matter-wave interference calculable; discarding their angles prematurely discards physically testable information. The Born rule example compares two measurement bases explicitly.

A probability distribution describes how the values of a random variable are allocated among possible outcomes. More precisely, the law of a random variable $X$ assigns each measurable set $B$ the probabilityIt is a probability measure on the values of $X$. The underlying experiment can be more elaborate than the output: different experiments may produce the same probability distribution. The law of a random variable retains the probabilities needed to answer questions about that output. ^[ref].

For a finite example, let $X$ take values $0,1,3$ with probability mass function $p_X(0)=0.2$, $p_X(1)=0.5$, $p_X(3)=0.3$. Its probabilities sum to one. The event $0<X\le 1$ has probability $0.5$, whereas $0\le X\le 1$ has probability $0.7$: including one endpoint matters when it carries positive probability.

For a contrasting random variable $Y$, choose the probability density function $f_Y(y)=2y$ on $0\le y\le 1$, and zero elsewhere. It integrates to one. Now $\mathbb{P}(1/4<Y\le 3/4)={\int }_{1/4}^{3/4}2ydy=1/2$, while every single outcome has probability zero. The probability density function can have height $f_Y(0.8)=1.6$: probabilities come from integrals over outcome ranges.

The cumulative distribution function answers all threshold questions for a real random variable, regardless of whether there are jumps or a probability density function. A quantile function reverses the direction of the question: given a cumulative probability level, find a threshold. This connects descriptive summaries to inverse transform sampling, which constructs a chosen probability distribution from a uniform distribution.

Let $N$ be a nonnegative integer-valued random variable, and let $Y_1,Y_2,\dots$ be nonnegative integer-valued independent and identically distributed random variables, with common probability generating function $G_Y$. Require the whole sequence to be independent of $N$, so $N,Y_1,Y_2,\dots$ are jointly independent. Set the random sumwith the empty sum equal to zero. Given $N=n$, the probability generating function of an independent sum yields the conditional expectation $\mathbb{E}[s^S\mid N=n]=G_Y(s{)}^n$. By the law of total expectation,The composition is valid because $|G_Y(s)|\le 1$. Merely making the $Y_j$ independent of each other does not preserve their conditional laws when $N$ depends on them. This is the probability generating function version of transform composition for an independent random sum.

For a Poisson distribution count with mean $\lambda$, substitution into $G_N(u)=\exp (\lambda (u-1))$ gives a compound Poisson distribution withIf the jumps have finite second moment, conditioning also gives mean $\mathbb{E}[S]=\lambda \mathbb{E}[Y]$ and variance $\mathrm{Var}(S)=\lambda \mathbb{E}[Y^2]$. The extra spread comes from randomness in the count as well as randomness in the jumps.^[ref]

For an original example, choose $N\sim \mathrm{Poisson}(2)$ and jumps taking values one and two with equal probability. Keep the independence assumptions above. ThenHere $\mathbb{E}[Y]=3/2$, $\mathrm{Var}(Y)=1/4$, and $\mathbb{E}[Y^2]=5/2$. The law of total expectation and law of total variance giveThus the aggregate is not itself a Poisson distribution, whose mean and variance would coincide.

To recover the probability mass function $p_n=\mathbb{P}(S=n)$ without repeated differentiation, use $G_S^{\prime }(s)=(1+2s)G_S(s)$ and compare coefficients:For example, $(p_0,p_1,p_2,p_3)=e^{-2}(1,1,3/2,7/6)$. As an independent check, expanding $e^{s+s^2-2}=e^{-2}{e}^s{e}^{s^2}$ gives

The plotted probability masses stop at 20, although the support of a probability distribution is unbounded. The omitted tail probability isThis model has bounded jumps and an unbounded count. Its entire probability generating function does give a finite moment-generating function at every real argument, but the general random sum composition only guarantees the stated $|s|\le 1$ domain.

Probability inequalities bound event probabilities from limited information, such as moments, without requiring a complete probability distribution. Markov's inequality uses a nonnegative random variable's expected value; Chebyshev's inequality uses a variance to bound deviations from a mean. The attraction is a guarantee that applies to many possible probability distributions. Their bounds can be conservative for a particular model, and distributions attaining equality show when the available information really cannot give more.

A probability measure is a nonnegative, countably additive measure with total mass one. Thus $\mathbb{P}(\Omega )=1$, and for pairwise disjoint events $A_j$,The complement of an event has probability $1-\mathbb{P}(A)$. The probability of a union of overlapping events needs a correction: $\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cap B)$. Counting the overlap once is the simplest instance of the inclusion-exclusion principle.

A zero-probability event need not be impossible as a member of the sample space. Under the continuous uniform distribution on $[0,1]$, every individual point has probability zero, while the whole interval has probability one. Countable additivity applies to countable unions, not arbitrary uncountable ones; there is no contradiction.

For a string of empirical distribution $q$, independence givesMultiplying this by the size of a type class gives its exact probability. The entropy bounds for a type class then implyThese bounds apply to one attainable empirical distribution, not automatically to a tail event containing many classes.^[ref] If $p(a)=0$ but $q(a)>0$, every string in that class is impossible and $D_e(q\Vert p)=+\infty$.

As an exact example, take an independent and identically distributed binary source with $p(1)=1/4$. Each balanced four-symbol string has probability $(1/4{)}^2(3/4{)}^2=9/256$. There are six such strings, soThe relative entropy is $D_e((1/2,1/2)\Vert (3/4,1/4))=\frac{1}{2}\ln (4/3)$. The exponential upper bound is $e^{-4D_e}=9/16=0.5625$, visibly larger than the exact probability. It captures the cost of an atypical frequency without supplying the exact finite-length prefactor.

For $n=40$, the exact probability of twenty ones iswhile the exponential upper bound is approximately $0.00317121$. At the source frequency $q(1)=1/4$, relative entropy is zero and the upper bound is one; the exact probability of ten ones is only about $0.144364$. A likely neighborhood of frequencies need not give a large probability to a single exact profile.

The striking cancellation is that there are about $e^{n{H}_e(q)}$ strings in a class, but each costs about $e^{-n[H_e(q)+D_e(q\Vert p)]}$. The information entropy terms cancel, leaving relative entropy as the rarity exponent. Summing over polynomially many classes preserves that leading exponential scale. This is the elementary counting mechanism behind finite-alphabet versions of Sanov theorem and many error exponents in hypothesis testing.

Probability sampling chooses observational units by a specified random mechanism with known inclusion probabilities. This separates design-based representativeness from the convenience of obtaining particular observations. Simple random sampling, stratified sampling and cluster sampling are different designs with different precision and cost properties.

A probability space is a measure space $(\Omega ,\mathcal{F},\mathbb{P})$ whose total measure is one. The sample space $\Omega$ contains the elementary outcomes; the sigma-algebra $\mathcal{F}$ contains the events to which the model assigns probabilities; the probability measure $\mathbb{P}$ supplies those values. The sigma-algebra lets us combine observable questions using complements and countable unions while remaining within the model.

For two independent uniform dice, take $\Omega =\{1,\dots ,6{\}}^2$, $\mathcal{F}=\mathcal{P}(\Omega )$, and give each ordered pair mass $1/36$. A random variable might record the sum, the larger result, or whether both results are even integers. These are different functions on the same sample space, rather than different elementary experiments. A probability distribution records the mass of the values of one such random variable and can forget information retained by the full probability space.

Probability theory describes uncertainty through a probability space: possible outcomes form a sample space, observable questions are events, and a probability measure assigns them numbers between zero and one. The Kolmogorov axioms make those assignments consistent. They let us combine information about several events, update it through conditional probability, and study random variables without listing every possible outcome separately.

For example, roll two dice whose results are independent random variables, each with a discrete uniform distribution on $\{1,\dots ,6\}$. There are 36 equally likely ordered outcomes. Only $(1,6),(2,5),\dots ,(6,1)$ have sum seven, so that event has probability $6/36=1/6$. The sum itself is not uniformly distributed: sum two has only one outcome. Choosing a probability distribution for the elementary outcomes is part of the model; equal likelihood does not follow merely from counting the named possibilities.

The subject becomes especially useful when complete outcome lists are enormous or infinite. Independence turns a joint calculation into simpler pieces, expected value summarizes a random variable, and the law of large numbers explains how averages can become predictable even when individual observations are uncertain. The central limit theorem describes the scale and shape of many remaining fluctuations. These are distinct claims: a small variance of an average does not make every observation close to its expected value.

There is a satisfying unity here: dice, hidden causes, and Brownian motion all use the same rules for events and conditional probability. A model still needs justification. Its exact deductions describe what its assumptions imply; comparison with observations belongs to statistical inference.

The probable error is the half-width of an interval around a specified central value containing probability $1/2$. For a normal distribution $X\sim N(\mu ,{\sigma }^2)$ with $\sigma >0$, its quantile function giveswhere $\Phi$ is the standard normal cumulative distribution function and $\sigma$ is the standard deviation. The factor is specific to the normal distribution. The centered Cauchy distribution with parameter $\gamma >0$ has probable error $\gamma$, despite having no finite variance. Thus a central probability width need not require a finite second moment.

For finitely many probability spaces $(X_i,{\mathcal{F}}_i,{\mu }_i)$, their product probability space has underlying set ${\prod }_i{X}_i$, the product sigma-algebra and product measure ${⨂}_i{\mu }_i$. The coordinate projections are jointly independent: for measurable $B_i\subseteq X_i$, the probability of all coordinate conditions is ${\prod }_i{\mu }_i(B_i)$. Conversely a family of independent coordinates has this product joint distribution. Product structure permits coordinate exposure and independent resampling in concentration arguments.

For independent counts with means $p_{i1}{\lambda }_i$ and $p_{i2}{\lambda }_{i}r$, let $t_i=y_{i1}+y_{i2}$. Maximizing the Poisson regression likelihood in each baseline rate gives the displayed estimate. The remaining log-rate-ratio score isEquivalently, conditional on $t_i$, the second count has a binomial distribution with success probability $r{p}_{i2}/(p_{i1}+r{p}_{i2})$. Each positive-total term increases strictly with $r$; if both treatment totals are positive, the score has a unique positive root. This reduces a many-parameter fit to a scalar equation.

A projective measurement has self-adjoint orthogonal projections $P_i$ satisfying $P_i{P}_j={\delta }_{ij}{P}_i$ and ${\sum }_i{P}_i=I$. Measuring an observable with discrete spectral decomposition $A={\sum }_i{a}_i{P}_i$ reports eigenvalue $a_i$ with Born rule probability $p_i=\mathrm{Tr}(P_i\rho )$. A projector can have rank greater than one: the outcome then identifies an eigenspace, rather than a particular eigenvector inside it. Projectors specify the outcome statistics; the usual ideal conditional update is the additional Lüders rule.

In the qubit example from measurement in quantum mechanics, $Z=|0⟩⟨0|-|1⟩⟨1|$. At $\theta =\pi /3$, the zero and one probabilities are $3/4$ and $1/4$, regardless of the relative quantum phase $\varphi$. Under the Lüders rule, obtaining zero prepares $|0⟩$; obtaining one prepares $|1⟩$, up to global phase. Repeating the same measurement returns the same label with certainty, provided no intervening evolution or operation changes the measured sector. Repeatability concerns the conditional output, and does not mean the first outcome was predetermined.

Change to $X$ and its Pauli X eigenstates $|\pm ⟩=(|0⟩\pm |1⟩)/\sqrt{2}$. Adding the two probability amplitudes before squaring givesAt $\theta =\pi /3$ and $\varphi =0$, $p_{+}=(2+\sqrt{3})/4\simeq 0.9330$ and $p_{-}\simeq 0.0670$. At $\varphi =\pi /2$, both are $1/2$; at $\varphi =\pi$, the first pair is exchanged. The $Z$ statistics remain $3/4,1/4$ throughout. This is the same addition-and-cancellation principle as wave interference: relative quantum phase changes how amplitudes reinforce or cancel in another basis.

The beauty of this basis dependence is that information invisible to one measurement becomes accessible to another. A $Z$ measurement of an ensemble estimates its quantum state populations, while $X$ probes one component of its quantum coherence. These are experiments on separate identically prepared copies. Sequentially measuring $Z$ and then $X$ on one copy instead probes the disturbed post-measurement state. Quantum state tomography combines suitable ensemble statistics to reconstruct the original state.

A proper scoring rule makes reporting the true probability distribution optimal in expected value, for every true probability distribution in the allowed class of forecasts. In the reward convention, the inequality in the title holds for every report $P$ and true law $Q$. The expected value is always taken under $Q$, because the report does not change how the outcome is generated. In the loss convention the same definition readsThis requirement makes a proper scoring rule useful for eliciting probabilities and comparing predictive distributions: deliberately distorting a known forecast cannot improve its expected reward or expected loss. The reward convention and its strict version are set out in Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others (Section 1).

For the Bernoulli distribution with $q=0.7$, the Brier score has expected loss $(p-0.7{)}^2+0.21$, and the negative logarithmic scoring rule has expected loss $-0.7\ln p-0.3\ln (1-p)$. Both are minimized at the honest report $p=0.7$. The linear probability score, by contrast, maximizes its expected reward at $p=1$. These are statements about averaging over both possible outcomes with their actual probabilities.

The left panel displays each excess expected loss divided by that rule's expected loss at $p=q$: $0.21$ for the Brier score and approximately $0.610864$ for the logarithmic scoring rule. This normalization permits comparison of curve shapes and preserves the minimizing report, since it subtracts a constant and divides by a positive constant independent of $p$. Zero means no excess expected loss. The right panel shows the unmodified linear probability score reward, with its maximizing direction labeled explicitly.

A hypergraph has Property B if its vertices of a hypergraph can be coloured with two colours so that every hypergraph edge meets both colour classes. An $r$-uniform hypergraph with fewer than $2^{r-1}$ edges has Property B, by a fair independent colouring and the union bound. Conversely, for $r\ge 2$, take $N=2r^2$ vertices and independently sample $m=\lceil (N\log 2+1)2^r\rceil$ uniformly random $r$-subsets. In any fixed colouring an edge is monochromatic with probability at least $2^{-r}$: convexity of binomial coefficients reduces to the balanced colouring, and $\left(\genfrac{}{}{0ex}{}{r^2}{r}\right)/\left(\genfrac{}{}{0ex}{}{2r^2}{r}\right)\ge 2^{-r-1}$. The union bound over $2^N$ colourings gives a probability at most $2^N{e}^{-m{2}^{-r}}<1$ of any proper colouring. Removing repeated edges leaves a hypergraph without Property B and with $O(r^2{2}^r)$ edges.

For positive total effort, a proportional allocation contest awards player $i$ the prize with probability $b_i/{\sum }_j{b}_j$, or gives it that proportion of a divisible prize. A convention is needed at zero total effort. This allocation is different from the discontinuous highest-effort rule of an all-pay auction.

A pseudorandom number is an output of a deterministic algorithm chosen to imitate samples from a specified probability law. For example, a linear congruential generator returns fractions $S_i/M$ on a finite grid in $[0,1)$. Their intended uniform distribution is an approximation, and their serial dependence must be assessed separately from the generator's period.