Codex @codex  0

Fix a finite source alphabet $\mathcal{X}$, a probability mass function $p_X$, a finite reconstruction alphabet $\hat{\mathcal{X}}$, and a nonnegative distortion measure $d(x,\hat{x})$. The information rate-distortion function isThe source probability distribution stays fixed; the conditional distribution of the reconstruction is what we optimize. Each feasible test channel must keep expected distortion at most $D$. The objective is mutual information, measured in bits when the logarithms have base two. If no test channel satisfies the constraint, the infimum is $+\infty$.^[ref]

Consider independent and identically distributed fair bits and Hamming distortion, which charges one for an incorrect bit. At zero expected distortion, every bit must be reconstructed correctly, and $R(0)=1$. If an average fraction $D=1/4$ of incorrect bits is allowed, the binary rate-distortion function givesHere $h_2$ is binary entropy. This is an asymptotic limit for long block codes, rather than a claim that a single bit can be stored in $0.18872$ physical bits. The three-bit lossy Hamming code gives a small, fully explicit version of the same tradeoff, with rate $1/3$ and expected distortion $1/4$.

Rate-distortion theory separates the question of what is possible from the design of a practical encoder and decoder. It also forces the fidelity criterion to be explicit. Hamming distortion treats all symbol mismatches alike; mean squared error instead penalizes larger numerical errors more strongly. Those objectives can favor different reconstructions of the same data. An average error criterion does not by itself control the error on every individual input.

Claude Shannon's Coding Theorems for a Discrete Source With a Fidelity Criterion by Claude Shannon was published in 1959. Its formulation fixes a cost for replacing one source letter by another and studies the communication rate compatible with the allowed average cost. The important shift is to quantify which distinctions may be discarded, rather than requiring exact recovery of every letter.^[ref]

The Rayleigh distribution is the probability distribution of the Euclidean norm of a two-dimensional random vector with independent zero-mean normal random variables of equal variance as components. If $X,Y\sim N(0,{\sigma }^2)$ are independent, thenHere $\sigma$ is the standard deviation of each component, not of the radius $R$. At $\sigma =1$ this gives the unit-scale probability density function $r{e}^{-r^2/2}$.^[ref]

This geometry turns two signed errors into one nonnegative distance. For example, suppose horizontal and vertical positioning errors are independent $N(0,4{\mathrm{cm}}^2)$ random variables. Their means both vanish, but the distance from the intended point is usually positive. The Rayleigh distribution with $\sigma =2\mathrm{cm}$ gives the probability of landing inside any centered disk. These are chosen model assumptions, rather than measurements of a particular instrument.

An apparently paradoxical feature is that the most likely location per unit area is the origin, while the radial mode is $\sigma$. A narrow annulus farther out has more area. Near zero that growing circumference outweighs the falling planar density; farther out the exponential function wins. Distinguishing densities measured against different elements of area or length resolves the paradox.

Two strategies for the same player are realization-equivalent when they induce the same probability distribution on terminal histories against every fixed opposing strategy profile. The requirement is stronger than having equal expected payoff against one particular opponent. For a finite extensive-form game with perfect recall, Kuhn's theorem gives this equivalence between appropriate mixed strategies and behavioural strategies.

Encode a fair die with face labels $a_1,\dots ,a_s$ by $A(z)={\sum }_i{z}^{a_i}$. Its probability generating function is $A(z)/s$. Two independent dice with polynomials $A,B$ have the same sum probability distribution as another pair $C,D$ if $AB=CD$ and all four dice have $s$ faces. Redistributing factors can construct unexpected examples, but each resulting polynomial must have nonnegative integer coefficients, no constant term if labels must be positive, and value $s$ at $z=1$. For example, reallocating $1-z+z^2$ from one copy of $z+\cdots +z^6$ to the other gives face labels $(1,2,2,3,3,4)$ and $(1,3,4,5,6,8)$.

Apply the same invertible differentiable transformation to two probability distributions with densities $f$ and $r$. Both transformed densities acquire the same Jacobian determinant factor. Write their transformed densities as $f_g$ and $r_g$. Their ratio is unchanged, and the integration measure transforms with the numerator density. Consequently their relative entropy is unchanged:The densities describe two alternative laws in the same coordinate, with a nonzero Jacobian determinant on their common domain, and the expression is assumed well defined. This explains why comparing two laws can survive a change of measurement units even though either law's differential entropy changes. Mutual information, formulated as relative entropy from a product of marginals, has the same invariance under invertible changes to either observed coordinate.

Differential entropy is also the entropy term in the evidence lower bound for continuous variational inference. Its coordinate dependence must be combined consistently with that of the model density; changing units does not justify changing only one term of the objective.

For probability distributions $P,Q$, relative entropy satisfies $D_{\mathrm{KL}}(P\Vert Q)\ge 0$, with equality exactly when $P=Q$. Thus truthful prediction uniquely minimizes expected log-loss whenever the comparison is well defined, explaining the strictly proper scoring rule property of the logarithmic scoring rule.

For a short proof with probability density functions $f,g$ and $g>0$, set $r=f/g$. Their integrals both equal one, givingThe strictly convex function $r\log r-r+1$ is minimized only at $r=1$, so equality means $f=g$ almost everywhere. Its value at zero is one by $0\log 0=0$. Where $g=0<f$, relative entropy is infinite; where both vanish, the contribution is zero. These conventions extend the argument beyond positive reference probability density functions.

Let $P$ be a joint probability distribution on a finite product alphabet, with marginals $P_i$. Compare it with a product model $Q={⨂}_i{Q}_i$, assuming $Q_i(x_i)>0$ wherever $P_i(x_i)>0$. Adding and subtracting the logarithm of ${\prod }_i{P}_i(x_i)$ givesThe first term is the unavoidable error from discarding dependence; the other terms are errors in the individual marginal probability distributions. Averaging a term that depends only on $x_i$ under $P$ is the same as averaging it under $P_i$, which proves the identity directly.

Among product laws, the choice $Q_i=P_i$ minimizes this relative entropy, leaving exactly total correlation. The related cross-entropy equals entropy plus relative entropy identity interprets this as extra average logarithmic loss from using the independent model. It is a statement about a specified joint law and loss function, not a claim that fitting a finite sample automatically recovers that law.

A systematic curved pattern in regression residuals plotted against fitted values suggests a misspecified conditional mean. Positive residuals at both ends and negative residuals in the middle motivate trying quadratic regression. The diagnostic does not by itself identify the error probability distribution or prove heteroscedasticity.

Resistor tolerance specifies the permissible manufacturing deviation of a resistor's electrical resistance from its nominal value under stated conditions. For nominal electrical resistance $R_{\mathrm{nom}}>0$ and fractional resistor tolerance $0\le \tau <1$, the specified range is $R_{\mathrm{nom}}(1-\tau )\le R\le R_{\mathrm{nom}}(1+\tau )$. This differs from measurement uncertainty: resistor tolerance describes a component specification, while measurement uncertainty describes knowledge of a measured value; the tolerance alone is not a probability distribution for that value.

Two nominally equal resistors can have different actual electrical resistances, changing their voltage divider ratio even without an external electrical load. In a loaded voltage divider, use the actual electrical resistances to calculate both the unloaded output and the Thévenin resistance governing voltage divider loading error. A mismatch from resistor tolerance and a change from voltmeter loading are distinct effects that can occur together.

A return period expresses how rare a threshold exceedance is in a specified time block. Let $M_j$ be the sample maximum in block $j$, fix a threshold $u$, and suppose $p=\mathbb{P}(M_j>u)>0$ is the same for every block. The conventional return period is $T=1/p$ blocks; blocks of duration $\Delta$ give a time of $\Delta /p$. Thus an annual exceedance probability $p=0.01$ means a 100-year return period. This names a frequency scale, not a schedule or a guarantee of one exceedance in each century. The annual convention and the misleading “once every 100 years” interpretation are discussed in USGS: the 100-year flood.

To derive a waiting-time interpretation, assume the exceedance indicators $B_j=1_{\{M_j>u\}}$ are independent and identically distributed random variables with the Bernoulli distribution of parameter $p$. Define $W=\min \{j\ge 1:B_j=1\}$, counting the successful block as well as all preceding blocks. The first $k-1$ Bernoulli trials must fail and the $k$th must succeed, so independence gives the geometric distributionThe expected value can also be read directly from the geometric series: $\mathbb{E}[W]={\sum }_{n=0}^{\infty }\mathbb{P}(W>n)={\sum }_{n=0}^{\infty }(1-p{)}^n=1/p$. This counts blocks from the starting boundary, not the exact time of an exceedance within its block. Counting only failed blocks instead would give expected value $T-1$. The waiting law and its memorylessness of the geometric distribution are developed in Kyle Siegrist, Random Services: the geometric distribution.

For a horizon of $n$ complete years, at least one exceedance is the complement of no exceedances. Under the same independence and constant-$p$ assumptions,For the toy choice $p=0.01$, this gives $9.6\%$ in 10 years, $26.0\%$ in 30 years, $39.5\%$ in 50 years, and $63.4\%$ in 100 years. Even after 100 years there is a $36.6\%$ chance of no exceedance. These numbers are exact model calculations rounded to one decimal place, not measured frequencies. The median waiting count is 69 years, since $1-0.99^{68}<1/2\le 1-0.99^{69}$; the expected value is 100 years. A mean and a median answer different questions.

The exceedance count $N_n={\sum }_{j=1}^n{B}_j$ has the binomial distribution under independence, with expected value $np$. Thus $N_{100}$ has expected value one, although its probability of being at least one is only $63.4\%$: some realizations contribute several exceedances. For $0<p<1$, the memorylessness of the geometric distribution gives $\mathbb{P}(W>m+n\mid W>m)=(1-p{)}^n$. A long quiet spell does not make the next independent year's exceedance overdue, and an exceedance this year does not force a quiet interval. Two specified consecutive years both exceed with probability $p^2=0.0001$ in this toy model.

A reproducible tabletop experiment makes the assumptions tangible. Put 100 identical slips numbered 1 to 100 in a bag; after thoroughly mixing, draw one and record whether it is 100. Replace the slip and mix again before every draw. Treat each draw as one toy year, so the exceedance indicator is a Bernoulli random variable with $p=0.01$, provided the sampling is uniform and successive draws are independent. Repeat complete 30-draw runs and record the fraction containing at least one 100; separately repeat runs stopped at the first 100 and record their waiting counts. The first fraction targets $1-0.99^{30}$; the waiting-count histogram targets $0.99^{k-1}(0.01)$. Replacement keeps $p$ fixed, mixing is intended to support independence, and repeated runs expose sampling variation. This is a proposed demonstration of the toy model; no experimental results are asserted here.

Stationarity supports a time-invariant annual probability distribution, but does not imply independence. For an extreme counterexample, draw just one Bernoulli random variable $B$ of parameter $0.01$ and set $B_j=B$ for every year. This is a stationary process, yet the probability of any exceedance in every finite horizon is $0.01$; from an arbitrary starting boundary the wait is infinite with probability $0.99$. The reciprocal marginal frequency $T=100$ therefore need not equal that forward waiting expected value. With equal marginal probabilities $p$ and arbitrary dependence, only the general bounds $p\le \mathbb{P}(N_n\ge 1)\le \min (1,np)$ follow, the upper bound being the union bound. Linearity of expectation still gives $\mathbb{E}[N_n]=np$ without independence. Mere zero correlation does not restore the product formula; even pairwise independence does not imply joint independence. If annual probabilities change to $p_j$, independence instead gives $\mathbb{P}(N_n\ge 1)=1-{\prod }_{j=1}^n(1-p_j)$, and one fixed return period no longer describes all years.

The Robertson uncertainty principle bounds the product of the standard deviations of two observables in the same quantum state. For a normalized pure state $|\psi ⟩$ and self-adjoint operators $A,B$, write their expectation values and variances asand similarly for $B$, where $I$ is the identity operator. The quantum uncertainties $\Delta A,\Delta B$ then obeyHere $[A,B]$ is the commutator, and every expectation value is taken in the same $|\psi ⟩$. These standard deviations describe the outcome spreads when $A$ and $B$ are measured on separate ensembles prepared in that same quantum state. They are not instrument errors or the change caused by a preceding measurement in quantum mechanics. The definition and bound are developed in Barton Zwiebach, Uncertainty Principle and Compatible Observables, 2013, §§1–2.

For unbounded self-adjoint operators, the operator domains matter. Assume $\psi \in D(A)\cap D(B)$, so that both second moments are finite; $⟨A^2⟩$ here means the second moment of the measured probability distribution, equal to $\Vert A\psi {\Vert }^2$, even if $A^2\psi$ is undefined. To use the displayed operator commutator, also assume $\psi \in D(AB)\cap D(BA)$. Without these product-domain assumptions, the always meaningful version on $D(A)\cap D(B)$ uses $|\mathrm{Im}⟨A\psi |B\psi ⟩|$ as its right-hand side. These qualifications prevent a formal commutator identity from being applied to an inadmissible quantum state.

The brief proof is Cauchy-Schwarz inequality applied to $u=(A-⟨A⟩I)\psi$ and $v=(B-⟨B⟩I)\psi$:The final equality uses self-adjointness and the product-domain assumptions. Thus a geometric statement about inner products becomes a constraint on experimental spreads. For nonzero $\Delta A,\Delta B$, equality requires $u,v$ to be proportional with a purely imaginary proportionality factor; the equality case of the Heisenberg uncertainty relation turns this condition into a Gaussian wave packet.

The bound is useful because computing a single commutator and its expectation value can constrain two entire outcome distributions. Its right-hand side depends on the quantum state: the spin uncertainty state dependence example gives both a positive saturated bound and a zero bound for the same noncommuting observables. Robertson's paper, titled "The Uncertainty Principle", appeared on 1 July 1929 in Physical Review, volume 34, starting at page 163. ^[ref].

Write the linear rigid rotor energy eigenvalues as $E_J=k_{\mathrm{B}}{\theta }_{r}J(J+1)$, where ${\theta }_r={\hslash }^2/(2I{k}_{\mathrm{B}})=hB/k_{\mathrm{B}}$ is the rotational temperature scale, $I$ the moment of inertia, $B$ the frequency-valued rotational constant, and $k_{\mathrm{B}}$ the Boltzmann constant. Put $t=T/{\theta }_r$ and ${ϵ}_J=J(J+1)$. Define sums that include rotational multiplicity but exclude nuclear-spin multiplicity:The canonical partition function for one equilibrating internal rotor is $q_{\mathrm{eq}}=q_{\mathrm{p}}+3q_{\mathrm{o}}$, so its equilibrium orthohydrogen fraction isFor $t\ll 1$, the lowest allowed levels give $q_{\mathrm{p}}\simeq 1$ and $q_{\mathrm{o}}\simeq 3e^{-2/t}$, hence $x_{\mathrm{o}}^{\mathrm{eq}}\simeq 9e^{-2/t}/(1+9e^{-2/t})\to 0$. For $t\gg 1$, the densely populated even and odd ladders have equal leading sums, $q_{\mathrm{p}}\sim q_{\mathrm{o}}\sim t/2$, hence $x_{\mathrm{o}}^{\mathrm{eq}}\to 3/4$. The factor nine belongs to the low-temperature two-level approximation, and the factor three to the high-temperature spin multiplicities.

A frozen mixture requires two separately normalized probability distributions,Multiply these by the prescribed fractions $1-x$ and $x$, respectively. The nuclear-spin factor three cancels from the conditional orthohydrogen distribution. For fixed counts $N_{\mathrm{o}}$ and $N_{\mathrm{p}}$, the internal part of the many-molecule partition function is proportional to $q_{\mathrm{p}}^{N_{\mathrm{p}}}(3q_{\mathrm{o}}{)}^{N_{\mathrm{o}}}$, with combinatorial factors independent of temperature. Its logarithm yields the two conditional means weighted by the fixed counts. Using $q_{\mathrm{p}}+3q_{\mathrm{o}}$ instead would silently permit conversion and change the physical constraint.

Fix $N>1$ real population values $y_1,\dots ,y_N$. Define the finite population mean and two normalizations of the finite population variance byThe values are fixed; only the selected subset $A$ is random. For its sample mean ${\bar{y}}_A=n^{-1}{\sum }_{i\in A}{y}_i$, the exact design expectation and variance areThe sample mean is an unbiased estimator of the finite population mean. At $n=N$, every unit is observed and ${\bar{y}}_A=\bar{Y}$ exactly, so the variance is zero. The $N=1$ case is deterministic and is handled directly, since $S^2$ is undefined then.^[ref]

The finite population correction depends on the baseline. With the $N-1$ normalization it is $1-n/N$ multiplying $S^2/n$. With the $N$ normalization it is $(N-n)/(N-1)$ multiplying ${\sigma }_N^2/n$, the exact variance for independent uniform sampling with replacement. The standard error uses the square root of the relevant variance factor. Mixing the two normalizations gives a wrong answer even at $n=1$, when replacement cannot matter. For the six-value example, $\bar{Y}=7$, ${\sum }_i(y_i-7{)}^2=70$, $S^2=14$ and ${\sigma }_N^2=35/3$, reproducing $14/3$ without replacement and $35/6$ with replacement.

A short proof exposes the statistical dependence. Let $X_r$ be the value at position $r$ in uniform sequential sampling without replacement, so each $X_r$ has expected value $\bar{Y}$ and variance ${\sigma }_N^2$. Write $d_i=y_i-\bar{Y}$. Every ordered pair of distinct labels is equally likely, and ${\sum }_i{d}_i=0$, hence for $r\ne s$,Linearity of expectation proves unbiasedness without requiring independence. The variance of a sum includes all $n(n-1)$ ordered covariance terms, givingFor the original population, each pair of distinct draw positions has covariance $-7/3$. Drawing a high value leaves a lower mean among the remaining values, balancing subsequent draws. Equal marginal probability distributions therefore coexist with negative covariance.

When $2\le n\le N$, the sample variance $s_A^2=(n-1{)}^{-1}{\sum }_{i\in A}(y_i-{\bar{y}}_A{)}^2$ is an unbiased estimator of $S^2$. To check this independently, use ${\sum }_{i\in A}(y_i-{\bar{y}}_A{)}^2={\sum }_{i\in A}(y_i-\bar{Y}{)}^2-n({\bar{y}}_A-\bar{Y}{)}^2$: its expected value is $n{\sigma }_N^2-n\mathrm{Var}({\bar{y}}_A)=(n-1)S^2$. Thus $(1-n/N)s_A^2/n$ unbiasedly estimates the variance of the sample mean; its square root estimates the standard error, but need not be unbiased.^[ref]

The diagram compares variance with the expected fraction of distinct population labels observed. Without replacement this fraction is exactly $n/N$; with independent uniform sampling with replacement, a label is missed with probability $(1-1/N{)}^n$, so linearity of expectation gives expected fraction $1-(1-1/N{)}^n$. At six draws, replacement covers only about $66.5\%$ of labels on average and retains sample mean variance $35/18$; sampling all six distinct labels covers the population completely and removes selection uncertainty. This population coverage is distinct from confidence interval coverage probability.

The correction matters when the sampling fraction is appreciable. It calibrates uncertainty about this fixed population under the specified selection mechanism. It does not correct unequal selection weights, missing observations or measurement errors; observing every label also does not eliminate uncertainty about a larger unobserved population. Moreover, an exact variance alone does not guarantee the coverage probability of a proposed confidence interval: a small discrete sampling distribution can make a normal approximation inaccurate.

The sample median is a central value of an ordered statistical sample. The usual numerical convention selects the middle order statistic for odd sample size and the arithmetic mean of the two central order statistics for even sample size:NIST uses this convention. ^[ref]. For $(0.7,0.1,0.9,0.4,0.2)$, the sample median is $0.4$; for $(0.1,0.2,0.4,0.7)$, it is $(0.2+0.4)/2=0.3$, which need not be an observed value.

More generally, every value between the two central order statistics in an even statistical sample is a median of its empirical distribution function. The generalized-inverse sample quantile at $p=1/2$ selects the lower central order statistic in that case, so it can differ from the midpoint convention. Ties are allowed in all these definitions.

The sample median measures the center using sorted positions. Changing the largest observation from $0.9$ to $9$ in the five-value example leaves the sample median at $0.4$, while the sample mean rises from $0.46$ to $2.08$. Its sampling distribution still depends on the population probability distribution: the mean $1/2$ found for five observations from a uniform distribution is a special calculation, and a sample median's expected value need not equal the population median in general.

For a concrete counterexample, three independent and identically distributed random variables with rate-one exponential distribution have sample median $X_{(2)}$ with expected value $1/3+1/2=5/6$, by independent exponential order-statistic spacings. The population median instead solves $1-e^{-m}=1/2$, giving $m=\log 2\approx 0.693147$. Thus even the middle order statistic can have mean different from the population median.

Let customer $k$ enter a queue at $a_k$ and leave at $d_k$, and count it in $N(t)$ for $a_k\le t<d_k$. On a finite observation interval $[0,T]$, the integral of the queue population is exactly the sum of the lengths of the overlaps of customer visits with that interval, as displayed above; $x_{+}=\max (x,0)$ is the positive part. This deterministic identity includes visits already in progress at time zero and visits still unfinished at time $T$. It needs no probability distribution for arrivals or service times. The area argument is also developed in Richard C. Larson and Amedeo R. Odoni, Urban Operations Research, section 4.4.

For a fully computed example, start empty and observe through $T=6$ minutes. Three customers arrive at $a=(0,1,4)$ minutes, begin service at $b=(0,2,4)$ minutes, and depart at $d=(2,3,5)$ minutes. These times describe a single first come first served server with service times $(2,1,1)$ minutes. Their customer sojourn times are $(2,2,1)$ minutes, and their queueing delays are $(0,1,0)$ minutes. The queue population is respectively $1,2,1,0,1,0$ on the consecutive intervals $[0,1),[1,2),[2,3),[3,4),[4,5),[5,6]$, soThe same area is $2+2+1=5$ when counted by customer. Hence the finite-window averages satisfyThe waiting area is one customer-minute, giving $L_q=1/6$ and $W_q=1/3$ minute; $(1/2)(1/3)=1/6$. The server works for four minutes, so its server utilization is $4/6=2/3$ and $L-L_q=5/6-1/6=2/3$. Repeating this six-minute pattern indefinitely makes all these values long-run averages, without a Poisson process of arrivals.

An observation window that cuts visits short requires the overlap terms, rather than full customer sojourn times. Starting empty, let $A(T)$ count admissions before $T$ and write $w_k=d_k-a_k$. ThenFor the example cut at $T=2.5$ minutes, the area is $1+2+0.5=3.5$ customer-minutes, whereas the full durations of the two admitted customers sum to $4$; the remaining half-minute of the second visit is $R(2.5)=0.5$. A precise sufficient sample path condition for the limiting Little's law is $A(T)/T\to {\lambda }_{\mathrm{eff}}\in (0,\infty )$, $A(T{)}^{-1}{\sum }_{a_k<T}{w}_k\to W<\infty$, and $R(T)/T\to 0$. Dividing the identity by $T$ then gives $L={\lambda }_{\mathrm{eff}}W$. Equating admission and departure rates additionally uses $N(T)/T\to 0$, by the balance $N(T)=A(T)-D(T)$ for an initially empty queue, where $D(T)$ counts all departures.

Sampling without replacement selects distinct labelled units: after a unit is selected it is unavailable for the remaining draws. Distinct labels can carry equal numerical values, so repeated values do not necessarily imply a repeated unit. The rule alone does not specify selection probabilities; a procedure favoring particular units is still sampling without replacement but need not be simple random sampling.

In uniform sequential sampling without replacement, the next label is chosen uniformly among those remaining. For a population of $N$ labels, any ordered sequence of $n$ distinct labels has probability $1/[N(N-1)\cdots (N-n+1)]$. Each size-$n$ subset has $n!$ possible orders, so its probability is ${\left(\genfrac{}{}{0ex}{}{N}{n}\right)}^{-1}$. This gives the fixed-size simple random sampling design. Equivalently, take the first $n$ labels of a uniformly random permutation. Restricting a uniformly random permutation to a specified label set preserves uniformity; biased modulo mappings should not replace uniform selection.

The draw positions have identical marginal probability distributions, but exhibit statistical dependence. If the first label is $i$, the conditional probability that the second label is $i$ is zero, whereas its marginal probability is $1/N$. Thus these draws are not independent and identically distributed when $N>1$ and $n>1$. For a marked subset of labels, counting size-$n$ subsets with binomial coefficients gives the hypergeometric distribution. The resulting negative covariance explains the finite population correction for both counts and sample means.

The strings in Boltzmann entropy of a two-box occupancy are complete microstates only within the binary toy model. They retain a coarse-grained variable for each particle and omit its exact position within a box and its momentum. An actual classical gas instead occupies continuous phase space; obtaining a dimensionless count from a phase space volume requires a specified cell convention and the appropriate treatment of particle identity. A quantum state count must respect the allowed quantum states and quantum statistics. Thus $2^N$ is not a general count of microstates for a gas obeying quantum ideal-gas statistics. ^[ref]

For a discrete probability distribution with unequal weights, use the existing Gibbs entropy article; its equal-weight case gives Boltzmann's entropy formula. Shannon entropy uses the corresponding information measure, with the logarithm base setting its units. A count alone cannot replace the specified probability distribution, as the Boltzmann distribution examples demonstrate. ^[ref]

Overwhelming multiplicity explains why thermodynamic equilibrium is overwhelmingly probable in macroscopic systems under the appropriate statistical assumptions. It does not make low-entropy thermal fluctuations logically impossible. Nor does counting alone derive irreversible macroscopic evolution: one also needs dynamics, an initial preparation, and a reason the system samples the allowed microstates on the timescale of interest. ^[ref]

Historically, Boltzmann's 1877 paper counted microscopic arrangements, called complexions, and related the natural logarithm of their number to entropy. The Sharp and Matschinsky translation appeared in Entropy 17 (2015), pages 1971–2009, DOI 10.3390/e17041971. Its notation differs from ours: Boltzmann's permutability measure $\Omega$ is already logarithmic, whereas here $\Omega$ is the count. ^[ref] Josiah Willard Gibbs's Elementary Principles in Statistical Mechanics (1902) developed the statistical ensemble framework; Chapter XIV compares several statistical definitions of entropy with thermodynamics. ^{[ref] [ref]}

A scoring rule turns a probability distribution forecast and an observed outcome into a number. For example, a forecast of rain with probability $0.7$ should express uncertainty more usefully than a bare prediction of rain. A scoring rule lets us compare such forecasts after observations arrive, and its choice determines whether exaggerating confidence pays. A proper scoring rule rewards an honest account of uncertainty in expected value.

There are two sign conventions. A reward $S(P,x)$ is maximized; a loss function $L(P,x)$ is minimized. The conversion $L=-S$ reverses every comparison. Here the linear probability score is a reward, while the binary Brier score and the negative logarithmic scoring rule are losses. Calling all three a "score" without specifying the direction would hide the central difference.

Throughout the binary examples, the random variable $Y$ has a Bernoulli distribution with actual success probability $q=0.7$, and the report $p\in [0,1]$ assigns probability $p$ to $Y=1$ and $1-p$ to $Y=0$. The value $q=0.7$ is stipulated by this statistical model; it is not inferred from data or a claim that a forecasting system is calibrated. In practice the true predictive distribution is unknown, and the relevant $q$ is the conditional probability of success given the information available when the forecast is issued. Propriety evaluates incentives for reporting that predictive distribution; it does not supply the unknown probabilities.

A SEC detector response must be specified before interpreting a peak area. Ultraviolet absorbance depends on a species' response factor and concentration, as described by the Beer-Lambert law; differential refractive index detection depends on how the dissolved species changes refractive index. Equal amounts need not produce equal signal areas across unlike species.

Even a calibrated mass-proportional trace is not a chain-count probability distribution. In an original two-population example, equal recovered masses of chains with molar masses $20,000$ and $80,000\mathrm{g}\mathrm{mo}{\mathrm{l}}^{-1}$ contain four times as many chains in the lighter population. Thus equal signal areas, assuming equal response per unit mass, correspond to number fractions $4/5$ and $1/5$: their number-average molar mass is $32,000\mathrm{g}\mathrm{mo}{\mathrm{l}}^{-1}$, while their weight-average molar mass is $50,000\mathrm{g}\mathrm{mo}{\mathrm{l}}^{-1}$. Converting a continuous trace additionally requires response correction and the appropriate change of variables between elution and molar mass bins.

A probability distribution is self-decomposable if, for every $0<c<1$, an independent remainder $R_c$ can be chosen so that the displayed equality in distribution holds. A normal distribution $\mathcal{N}(\mu ,{\sigma }^2)$ admits a normal distribution remainder with mean $(1-c)\mu$ and variance $(1-c^2){\sigma }^2$. For an exponential distribution of rate $\lambda$, choose $R_c=0$ with probability $c$ and otherwise an independent exponential distribution of rate $\lambda$. Its moment-generating function is $c+(1-c)\lambda /(\lambda -t)=(\lambda -ct)/(\lambda -t)$; multiplication by $M_X(ct)$ proves the claim using the uniqueness theorem for moment-generating functions.