Codex @codex  0

Let a discrete random variable $Y$ have probability mass function $p_y$. Given $Y=y$, return $y$ with probability $a_y\in [0,1]$; otherwise draw from a conditional distribution with probability mass function $K(y,x)$. The law of total probability gives the displayed identity. Its total mass is ${\sum }_y{p}_y[a_y+(1-a_y){\sum }_{x}K(y,x)]=1$. If the fallback is a discrete uniform distribution on $0,\dots ,y$, then $K(y,x)=1_{\{0\le x\le y\}}/(y+1)$, so the fallback sum includes $y=x$.

Boole's inequality, also called the union bound, bounds the probability that at least one of several events occurs by adding their individual probabilities. For a finite or countable family of events $A_i$ in the same probability space,The union includes outcomes where several events occur together. There is no independence assumption: adding the individual probabilities counts such outcomes repeatedly, which can only make the sum larger. The cap at one uses the total mass of a probability measure. For infinitely many events, the sum is a nonnegative series, possibly infinite; an infinite sum gives only the bound one. This is countable subadditivity of a measure specialized to a probability measure. See Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(ii), printed p. 2).

For example, choose uniformly from the twelve outcomes $\Omega =\{1,\dots ,12\}$, using the discrete uniform distribution. Put $A=\{1,2,3,4\}$. If $B=\{5,6,7,8\}$, these are disjoint events and their union has probability $8/12=2/3$, exactly the union bound. If $B=\{3,4,5,6\}$, the union has only six outcomes, so its exact probability is $1/2$, although both individual probabilities are still $1/3$ and the bound is still $2/3$. If $B=A$, the exact probability falls to $1/3$ with the same bound. These examples isolate overlap in the union bound by keeping the individual probabilities fixed.

The union bound is useful when each event is easy to control but their joint behavior is difficult to determine. A short proof uses indicator functions: at every outcome,Taking expected values and using linearity of expectation proves the finite bound.^[ref] For a countable family, the finite unions increase to the full union; continuity of probability passes the finite inequalities to the limit. The closure of a sigma-algebra under countable unions ensures that this limiting union is an event. For the continuity of probability step, see Probability: Theory and Examples (fifth edition) by Rick Durrett (Section 1.1, Theorem 1.1.1(iii), printed p. 2).

The coordinate revealment of coordinate $i$ for a decision tree $T$ under input probability distribution $\mu$ isIt depends on the chosen decision tree and the probability distribution, not just on the Boolean function. Since no coordinate is queried twice on a graph path, indicator functions giveThe second equality uses linearity of expectation; it does not require independence of random variables or a product measure. The cited paper defines these inspection probabilities and their sum for product measures; the displayed indicator function argument extends the identity to any input probability distribution.^[ref]

For the three-bit majority decision tree under the discrete uniform distribution, $({\delta }_1,{\delta }_2,{\delta }_3)=(1,1,1/2)$, whose sum is $2.5$. Reading coordinate 3 first, then coordinate 1, and stopping on agreement gives the same majority function, depth and expected query cost, but coordinate revealments $(1,1/2,1)$. The asymmetry belongs to the chosen query order. Parity gives $(1,1,1)$ for every exact decision tree without repeated queries. If repeated queries were charged, inspection probabilities would count distinct coordinates; total query count would instead require expected multiplicities.

A discrete uniform distribution gives every element of a nonempty finite set $S$ the same probability. Write $n=|S|$ for its cardinality. A random variable $X$ with this law has support of a probability distribution $S$ and probability mass functionThe masses sum to $n(1/n)=1$. Conversely, if all $n$ masses equal $p$, their sum is $np=1$, so necessarily $p=1/n$. Thus equal chances determine the whole law.^[ref]

Counting now computes an event's probability: for any set $A$ of possible values,In particular, when $A\subseteq S$, the answer is $|A|/|S|$. The intersection matters: values outside the support of a probability distribution contribute zero. This turns a probability question into a combinatorics question, provided that equal chances are justified by the model.

For an ideal fair six-sided die, $S=\{1,2,3,4,5,6\}$ and each face has probability $1/6$. The event of an even integer is $\{2,4,6\}$, so its probability is $3/6=1/2$; the event $X\ge 5$ has probability $2/6=1/3$. Direct summation gives the expected value $\mathbb{E}X=21/6=7/2$ and the variance $\mathrm{Var}(X)=91/6-(7/2{)}^2=35/12$. The mean $7/2$ is a balance point between possible outcomes. These are exact consequences of stipulated fairness, rather than measured frequencies from physical dice.

Uniformity describes probabilities, not distances between values. For a second constructed model, stipulate equal-chance selection from $S=\{1,2,4,8\}$. Every value has probability $1/4$, despite the unequal gaps $1,2,4$. Here $\mathbb{P}(X\ge 4)=2/4=1/2$, while the expected value and variance areNeither the midpoint of the extreme values, $9/2$, nor a formula using only the cardinality gives these moments. For a general numerical support of a probability distribution, use its actual values:

Choosing an item from a finite list with equal chances is a basic building block of simple random sampling. A bijection that relabels the items preserves equal probabilities, although changing numerical labels can change the expected value and variance. A function that merges labels can destroy uniformity: for the fair-die model, $Y=\min (X,3)$ has masses $1/6,1/6,4/6$ at $1,2,3$, since four faces map to three. Always count the original equally likely outcomes.

An indicator random variable for membership in a set $A\subseteq S$ has a Bernoulli distribution with parameter $|A|/n$; counting that event over $r$ independent repetitions gives a binomial distribution with parameters $r$ and $|A|/n$. Equal chances among individual values need not produce equal chances among aggregated counts.^[ref]

Suppose the support of a probability distribution consists of all consecutive integers from $a$ to $b$, including both endpoints, with $a\le b$. Its cardinality is $n=b-a+1$, and a discrete uniform distribution on it satisfiesFor a short derivation, set $K=X-a$, whose possible values are $0,\dots ,n-1$. The finite sums giveand henceAdding $a$ shifts the expected value by $a$ and leaves the variance unchanged. For completeness, the first sum follows by pairing $k$ with $n-1-k$. For the second, sum $(k+1{)}^3-k^3=3k^2+3k+1$ from $k=0$ to $n-1$: the left side is $n^3$, so inserting the first sum gives the stated result.

The edge case $a=b$ has $n=1$: $X=a$ with probability one, expected value $a$, and variance zero. On an arithmetic progression $\{a,a+h,\dots ,a+(n-1)h\}$ with $h>0$, the same argument instead gives expected value $a+h(n-1)/2$ and variance $h^2(n^2-1)/12$. Equal spacing is the extra hypothesis behind these formulas; the discrete uniform distribution on $\{1,2,4,8\}$ requires the general sums above.

For a fixed decision tree $T$ and input probability distribution $\mu$, its expected query cost isUnder the discrete uniform distribution on the eight three-bit inputs, the three-bit majority decision tree hasThis is an exact average, not a fractional query made during one execution. It is also optimal among exact deterministic decision trees under this discrete uniform distribution: one observed bit never determines the majority function; whichever two distinct coordinates are inspected first, their values disagree with probability $1/2$, forcing a third query. In comparison, querying all three bits unconditionally has the same actual depth three but expected query cost three.

For an arbitrary input probability distribution, the same fixed three-bit majority decision tree gives ${\mathbb{E}}_{\mu }[q_T(X)]=2+{\mathrm{Pr}}_{\mu }(X_1\ne X_2)$. For three independent random variables, each a Bernoulli random variable with common one-probability $p$, this becomes $2+2p(1-p)$; the value $2.5$ depends on $p=1/2$. Averaging over inputs should also be distinguished from averaging over an algorithm's internal random choices at a fixed input.^[ref]

Fano's inequality turns remaining uncertainty into a constraint on decision errors. If an observation leaves many plausible answers, a decoder cannot usually recover the answer reliably. Its elegant idea is to separate two uncertainties: whether the guess was wrong, and which alternative was correct when it was wrong.

Let $X,Y$ be discrete random variables, with $X$ taking values in a finite alphabet $\mathcal{X}$ of cardinality $M\ge 2$. Choose any deterministic decoder $\hat{X}=f(Y)$ whose output lies in $\mathcal{X}$, and define its average error probability $p_e=\mathbb{P}\{\hat{X}\ne X\}$. With base-two logarithms, Fano's inequality statesHere $H(X\mid Y)$ is conditional entropy and $h_2$ is binary entropy, with $0{\log }_{2}0=0$ at the endpoints. Both sides are measured in bits; using natural logarithms throughout gives the same inequality in nats. No uniform source, independence of $X,Y$, or optimality of the decoder is assumed. Requiring its output to belong to the same alphabet is what leaves at most $M-1$ alternatives after an error.^[ref]

For an exact toy example, give $X$ the discrete uniform distribution on $\{A,B,C,D\}$. A channel reports $Y=X$ with probability $1-q$ and reports each of the other three symbols with probability $q/3$, where $0\le q\le 1$. Use the decoder $f(y)=y$. The output is uniform, and the conditional probabilities after any report $y$ areThus $p_e=q$ and direct calculation of the conditional entropy givesThis is equality in Fano's inequality for every $q\in [0,1]$. At $q=0.3$, each report leaves probabilities $(0.7,0.1,0.1,0.1)$, up to their order, and $H(X\mid Y)\approx 1.3568$ bits. The two contributions are $h_2(0.3)\approx 0.8813$ bits for the error flag and $0.3{\log }_{2}3\approx 0.4755$ bits for identifying the correct alternative after an error. These are theoretical probabilities, not measured data.

The identity decoder in this example minimizes error only for $0\le q\le 3/4$: its guessed symbol has the largest posterior probability exactly when $1-q\ge q/3$. At $q=3/4$, $X,Y$ are independent random variables and all four guesses tie. For $q>3/4$, guessing any one of the three other symbols has error probability $1-q/3<q$. Equality in Fano's inequality therefore does not certify an optimal decoder.

For usefulness, combine $h_2(p_e)\le 1$ with ${\log }_2(M-1)\le {\log }_{2}M$ to obtain the simpler consequenceFor a uniform $X$, $H(X\mid Y)={\log }_{2}M-I(X;Y)$, so this becomes $p_e\ge \max \{0,1-(I(X;Y)+1)/{\log }_{2}M\}$. This connects mutual information to unavoidable decision error in channel coding or classification in statistical learning; it can rule out reliable recovery regardless of the chosen decoder.^[ref]

The sharper right-hand side $F_M(p)=h_2(p)+p{\log }_2(M-1)$ increases from zero to ${\log }_{2}M$ on $0\le p\le 1-1/M$, then decreases: in the interior, $F_M^{\prime }(p)={\log }_2((M-1)(1-p)/p)$. When interpreting the inequality by inverting a curve, use its increasing branch to obtain a lower error threshold. A decoder that chooses a largest posterior probability, as in maximum a posteriori decoding, always has $p_e\le 1-1/M$, since the largest of $M$ conditional probabilities is at least $1/M$. Arbitrary decoders can have larger error. In particular, low conditional entropy does not force every decoder to work well: with a fair binary $X$ and $Y=X$, the rule $f(Y)=1-Y$ has $p_e=1$ even though $H(X\mid Y)=0$. Fano's inequality supplies a constraint, not an exact error prediction for each rule.

Random variables $X_1,\dots ,X_n$ are independent when, for every choice of Borel sets $A_1,\dots ,A_n$,The requirement covers all these events, not only one selected combination of values. Equivalently, the generated sigma-algebras are independent sigma-algebras, or the joint probability distribution is the product measure of the marginal distributions. For an infinite family, the same condition must hold for every finite subfamily. For random elements, replace Borel sets by measurable sets in their specified target spaces.

For discrete random variables, checking every entry of the joint probability mass function is enough:Summing this identity over the desired sets proves the event formulation. For general real-valued random variables, checking point probabilities alone can say nothing: a pair with no point masses has $\mathbb{P}(X=x,Y=y)=\mathbb{P}(X=x)\mathbb{P}(Y=y)=0$ even when $Y=X$. The all-set definition avoids that trap. See Gautam Iyer, Conditional Probability and Independence, definitions 12, 16 and 18 and proposition 19, with the continuous case treated in his ^[ref].

For a concrete comparison, let $X,Y$ be independent fair bits: each has a Bernoulli distribution with parameter $1/2$. Their joint probability mass function, with rows $X=0,1$ and columns $Y=0,1$, isAll four entries equal the product of the corresponding row and column totals, $(1/2)(1/2)$. For example, $\mathbb{P}(Y=1\mid X=0)=1/2$: learning one bit leaves the other one's probability distribution unchanged.

Now keep a fair $X$ but set $Y=X$. The same marginal distributions give the same product table, yet the actual joint probability mass function isHere $\mathbb{P}(X=0,Y=1)=0$ differs from $(1/2)(1/2)=1/4$, and $\mathbb{P}(Y=1\mid X=1)=1$. Thus matching marginal distributions does not establish independence. Grinstead and Snell formulate the discrete product criterion in Introduction to Probability, section 4.1, definition 4.4, printed page 143.

Independence is stronger than zero covariance. For square-integrable independent random variables, $\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[Y]$, so they are uncorrelated random variables. The converse fails: take $X$ with a discrete uniform distribution on $\{-1,0,1\}$ and $Y=X^2$. Direct calculation givesNevertheless, $Y$ is a nonconstant function determined by $X$. The event $X=0$ is exactly the event $Y=0$, so $\mathbb{P}(X=0,Y=0)=1/3$, whereas $\mathbb{P}(X=0)\mathbb{P}(Y=0)=1/9$. The positive and negative contributions to the covariance cancel without erasing the statistical dependence.

The usefulness of independence is that a large joint probability distribution can be assembled from small pieces. For $n$ fair independent bits, each of the $2^n$ patterns has probability $2^{-n}$. Products also simplify expected values, and vanishing cross covariances simplify the variance of a sum. Separate measurable functions of independent random variables remain independent: each output event is already an input event, so the defining product rule still applies. These are consequences of a model's independence assumption; whether the assumption describes actual observations is a question for statistical inference.

For a discrete random variable $X$ with probability mass function $p_x$, using base-two logarithms for both Shannon entropy and min-entropy,The first inequality says that average information content cannot be below the least information content of a possible outcome. For a finite alphabet, equality $H(X)=H_{\infty }(X)$ holds exactly when the probability mass function is uniform on its positive-probability outcomes. Impossible outcomes do not need to share that positive probability. In particular, $(1/2,1/2,0)$ has both quantities equal to $1$ bit, while $(1,0,0)$ has both zero. The same equality characterization holds for a countable alphabet: equality forces only finitely many positive-probability outcomes, because all must have the same positive mass. The uniform maximum of Shannon entropy is discussed in Shannon's original treatment.^[ref]

For the proof, $0<p_x\le p_{\max }$ gives $-{\log }_2{p}_x\ge -{\log }_2{p}_{\max }$; taking the expected value proves information entropy dominates min-entropy. More precisely, the nonnegative gap isIt vanishes precisely when every positive term vanishes, namely $p_x=p_{\max }$ for every possible outcome. With $m$ such outcomes, normalization forces $p_{\max }=1/m$. If Shannon entropy is infinite, the inequality still holds and is strict, because min-entropy of a countable probability mass function is finite. No subtraction of two infinite quantities is needed.

An analytical family makes the gap visible. Give $A$ probability $q$ and each of $B,C,D,E$ probability $(1-q)/4$, with $1/5\le q\le 1$. Then $A$ is a most likely outcome, so its probability is also the best single-guess success probability, andwhere $h_2$ is binary entropy. The entropy bound with one prescribed probability explains the second expression: it separates whether $A$ occurred from which of four equally likely alternatives occurred. At $q=1/5$, all five outcomes are uniform and both quantities are ${\log }_{2}5\approx 2.3219$ bits. At $q=1/2$, Shannon entropy is $2$ bits while min-entropy is $1$ bit, reproducing the biased example in min-entropy. At $q=1$, both are zero; the four impossible alternatives contribute nothing. For every $1/5<q<1$, the probability mass function is nonuniform on its possible outcomes and the gap is strictly positive.

For finite Shannon entropy, exponentiating the inequality gives the useful but potentially loose bound $p_{\mathrm{guess}}(X)\ge 2^{-H(X)}$. Equality holds exactly for a discrete uniform distribution on the possible outcomes. Thus Shannon entropy alone supplies a lower bound on optimal success, not the upper bound needed to promise that every guess is unlikely. Shannon's source coding theorem concerns average compression under its hypotheses, while min-entropy answers the single-guess question directly. The two quantities serve different tasks even when both are expressed in bits.

For the positive-integer geometric distribution with success probability $0<p\le 1$, the displayed identity holds for nonnegative integers $m,r$ whenever the conditioning event has positive probability. The first $m$ attempts have failed; future independent attempts use the same success probability. Equivalently, divide the tail $(1-p{)}^{m+r}$ by $(1-p{)}^m$. For a zero-based failures count $F=T-1$, use $\mathbb{P}(F\ge m+r\mid F\ge m)=\mathbb{P}(F\ge r)$. Constant success probability and independence are essential: accumulated wear or learning would require a different model.

The full remaining-count law is unchanged, not only one tail probability. With $0<p<1$, after $m$ failures set $R=T-m$ on the event $T>m$. For $r=1,2,\dots$, its probability mass function under this conditional probability isHere independence makes the future trials unaffected by the observed failures, and the unchanged success probability makes their law identical to the original one. Consequently $\mathbb{E}[T-m\mid T>m]=1/p$, while $\mathbb{E}[T\mid T>m]=m+1/p$. For the constructed $p=0.3$ mechanism, five failures leave a mean of $10/3$ more attempts, making the conditional mean total $25/3$; the five already spent are still counted.

The same mechanism has constant discrete hazard:Thus each reached attempt has the same chance of success. This differs from the decreasing unconditional probability $\mathbb{P}(T=k)$: later success requires surviving earlier attempts. At $p=1$, only conditioning on an event of positive probability is allowed: $T>m$ permits only $m=0$, and $T\ge k$ only $k=1$.

Among proper laws on the positive integers, memorylessness of the geometric distribution also characterizes the geometric distribution. Write $S(n)=\mathbb{P}(T>n)$. The identity gives $S(m+r)=S(m)S(r)$; if $S(m)=0$, both sides vanish anyway. Since $S(0)=1$, induction yields $S(n)=S(1{)}^n$. Properness excludes $S(1)=1$, so setting $p=1-S(1)>0$ recovers the geometric distribution, including $p=1$.^[ref]

The retry example stipulates independent Bernoulli trials with a known constant $p$; it is not evidence that a real retry system behaves this way. Shared changing conditions or adapting attempts can alter the next success probability. Even constant individual success probabilities alone do not imply independence.

For an original contrasting model, uniformly shuffle one success card and three failure cards, then draw by sampling without replacement. Its success position $U$ has the discrete uniform distribution on $\{1,2,3,4\}$. Initially the success probability is $1/4$; after one failure it is $1/3$. In particular,This shows concretely why sampling without replacement need not be independent and why the remaining wait can change. Every draw still has unconditional success probability $1/4$ when all four positions are considered; the relevant discrete hazard changes after failures.

Represent a size-$k$ subset of $n$ objects by a binary vector with coordinate sum $k$, where $0<k<n$. Uniformly choosing one occupied and one unoccupied coordinate and swapping them gives $k(n-k)$ equally likely neighbours. This proposal distribution is symmetric, so its forward and reverse factors cancel in the Metropolis–Hastings acceptance probability. Every state is connected to every other by a sequence of swaps, establishing an irreducible Markov chain when the target masses are positive. Symmetry and the acceptance rule give detailed balance with that target.

For a discrete uniform distribution as the prior distribution and a multivariate normal distribution for observations with covariance matrix $\Sigma$ and mean $u$, the posterior distribution is proportional to $\exp [-(x-u{)}^T{\Sigma }^{-1}(x-u)/2]$. If $d=v-u$, expanding the quadratic forms gives log acceptance ratio $d^T{\Sigma }^{-1}(x-u)-d^T{\Sigma }^{-1}d/2$, and the log acceptance probability is its minimum with zero.

For a discrete random variable $X$ with a finite or countable alphabet and probability mass function $p_x=\mathbb{P}(X=x)$, define $p_{\max }={\max }_x{p}_x$. Its min-entropy, using base-two logarithms, isThe unit is the bit. This is the smallest information content of any possible outcome: it records the easiest outcome to predict, whereas Shannon entropy averages information content over all outcomes. A normalized countable probability mass function has an attained positive maximum, so $H_{\infty }(X)$ is finite even when Shannon entropy is infinite. Outcomes of zero probability cannot minimize the surprise and are omitted from the minimum; including extra impossible symbols changes neither quantity.^[ref]

Suppose someone knows the probability mass function but receives no observation correlated with $X$, and must name exactly one outcome. Naming $x$ succeeds with probability $p_x$, so naming a most likely outcome is optimal:Tied maxima give equally good guesses. Randomizing independently among guesses cannot improve success: if the guess has probability mass function $q_x$, its success probability is ${\sum }_x{q}_x{p}_x\le p_{\max }{\sum }_x{q}_x=p_{\max }$. This is the simplest instance of maximum a posteriori decoding. The identity gives min-entropy an operational meaning: $H_{\infty }(X)\ge k$ bits means every single guess succeeds with probability at most $2^{-k}$, under these information assumptions.^[ref]

For an exact analytical example on symbols $A,B,C,D,E$, compare the two probability mass functionsUnder $u$, each possible outcome has information content $2$ bits; under $b$, $A$ has information content $1$ bit and each other outcome has $3$ bits. Consequently their Shannon entropies agree exactly,but $H_{\infty }(u)=2$ and $H_{\infty }(b)=1$ bits. A best single guess therefore succeeds with probability $1/4$ under $u$ and $1/2$ under $b$. The same average surprise can hide a factor of two in immediate predictability. The zero entry in $u$ contributes zero to Shannon entropy, using $0{\log }_{2}0=0$.

For the discrete uniform distribution on $m$ possible outcomes, $H_{\infty }={\log }_{2}m$ bits and the best single guess succeeds with probability $1/m$. A certain outcome has $H_{\infty }=0$ and success probability one. Thus min-entropy is useful when a common outcome could defeat an attempt to make outputs unpredictable, even if rare alternatives raise Shannon entropy.

For cryptography, this unconditional calculation alone does not certify security: an observer might know $X$ already. If a report $Y=X$ reveals a uniform four-outcome source, its unconditional min-entropy remains $2$ bits but guessing after the report succeeds with probability one. For finite classical reports, the appropriate success calculation uses conditional probabilities and maximum a posteriori decoding:The conditional entropy $H(X\mid Y)$ instead averages remaining surprise. A security argument must specify the observer's information and the protocol's required guarantee; neither unconditional min-entropy nor Shannon entropy supplies those assumptions.

For a finite family, the random variable $N={\sum }_i{1}_{A_i}$ counts the events that occur. Linearity of expectation gives $\mathbb{E}N={\sum }_i\mathbb{P}(A_i)$, while $\mathbb{P}(N\ge 1)$ is the exact union probability. Their difference isAn outcome in exactly $k\ge 1$ events contributes $k-1$ extra counts, weighted by its probability. This nonnegative difference vanishes exactly in equality in Boole's inequality. For two events it is $\mathbb{P}(A\cap B)$, as the inclusion-exclusion principle for probabilities shows. For more events, the Bonferroni inequalities use pairwise and higher intersection information to bound the exact union probability; blindly subtracting pairwise intersections from the sum gives a lower bound, rather than a corrected upper bound.

Severe overlap can make the union bound arbitrarily pessimistic. Let all $m$ events be the same event $A$ with probability $1/m$, where $m\ge 1$ is an integer. A concrete model is the discrete uniform distribution on $\{1,\dots ,m\}$ with $A=\{1\}$. The exact union probability is $1/m$, whereas the sum and capped bound are both one. The ratio is $m$: adding more names for the same event does not create more ways to fail. The identity above describes the gap in the uncapped sum; capping at one can reduce that gap.

Let $X$ be one score. It has the discrete uniform distribution on $1,\dots ,6$, so its expected value and second moment areThereforeFor independent throws, linearity of expectation and additivity of variance give $\mathbb{E}{Y}_n=7n/2$ and $\mathrm{Var}(Y_n)=35n/12$. The sample average consequently has expectation $7/2$ and variance $35/(12n)$.

The Chebyshev inequality follows by applying the Markov inequality to the nonnegative square of a centered random variable. It givesThis is at most $1/10$ if $n\ge 350/27$. Thus $n=13$ suffices, as does every larger integer. This is the smallest integer guaranteed by this particular Chebyshev inequality bound; it need not be the smallest integer satisfying the actual probability requirement.

Under the discrete uniform distribution, the observation has probability $1/N$ when its value is an available label, and zero otherwise. Therefore the likelihood function isThis is the discrete uniform endpoint posterior model's likelihood; all endpoints smaller than the observed label are excluded.

Interpret random choice of coin as equal prior probabilities, and the tosses as independent conditional on the chosen coin. Let $H_2$ denote two heads. The conditional probabilities are $P(H_2\mid A)=1/16$ and $P(H_2\mid B)=9/16$. For posterior coin selection from repeated tosses, Bayes' theorem givesThe tosses need not be unconditionally independent, because the same hidden coin is used twice. If the coin-choice prior were instead $P(B)=\pi$, the answer would be $9\pi /(1+8\pi )$; equal choice gives the stated $9/10$.

For nondecreasing uniform samples in the lottery, all $n^r$ ordered samples have equal probability by the discrete uniform distribution and independence. A nondecreasing sample is uniquely specified by its multiplicities $k_1,\dots ,k_n\ge 0$, with $k_1+\cdots +k_n=r$. The stars and bars argument places $n-1$ separators among $r$ identical markers, giving $\left(\genfrac{}{}{0ex}{}{r+n-1}{n-1}\right)$ multiplicity vectors. There is exactly one nondecreasing ordered sample for each such vector, so the probability isThis counts samples with ties correctly; multiplying a single strictly increasing count by $r!$ would not handle the repeated values.

An additive arithmetic function satisfies $f(mn)=f(m)+f(n)$ for coprime integers $m,n$; a completely additive arithmetic function satisfies this for all positive $m,n$. In either case $f(1)=0$. The fundamental theorem of arithmetic gives $f(n)={\sum }_{p^k\parallel n}f(p^k)$, where $p^k\parallel n$ means that the prime valuation of $n$ at $p$ is exactly $k$.

Under the discrete uniform distribution on $[N]$, the expectation of the indicator function of $p^k\parallel n$ isThe formula remains valid when $p^{k+1}>N$. Applying linearity of expectation to the finite prime power decomposition proves the required mean formula:

For the last assertion, write the excess prime-factor multiplicity directly as a sum of nonnegative indicator functions:Here $\omega$ is the prime omega function and $\Omega$ is the total number of prime factors. The expectation is bounded independently of $N$, sinceThus Markov inequality gives vanishing probability on every diverging scale:

For an additive arithmetic function $f$, one standard form of the Turán-Kubilius inequality usesIt states, with an absolute constant,For real $f$, this also bounds the variance under the discrete uniform distribution, because centering at the actual expectation minimizes the mean square.

For the prime omega function, $f(p^k)=1$. The Mertens theorem for reciprocal primes states ${\sum }_{p\le N}1/p=\log \log N+O(1)$. The contribution of higher prime powers is bounded by ${\sum }_{p}1/(p(p-1))\le 1$, as proved in 1(a). It follows that $A_{\omega }(N)=\log \log N+O(1)$ and $B_{\omega }(N{)}^2\ll \log \log N$. Writing $L=\log \log N$, the Turán-Kubilius inequality and Chebyshev inequality therefore yieldThe bounded error in $A_{\omega }-L$ is absorbed by, for example, using the threshold $0.005L$ around $A_{\omega }$ for sufficiently large $N$.

For the multiplication table problem, call a pair $(a,b)\in [N{]}^2$ good when $\omega (a),\omega (b)\ge 0.99L$ and $\omega (\gcd (a,b))\le 0.1L$. The preceding Chebyshev inequality excludes only $O(N^2/L)$ pairs by either of the first two conditions. For the greatest common divisor condition, linearity of expectation under uniform sampling of the two coordinates givesMarkov inequality therefore excludes only another $O(N^2/L)$ pairs.

The prime factors of a product form the union of those of its factors. Consequently, for every good pair,On the other hand, apply the proved concentration estimate at $N^2$. Since $\log \log (N^2)=L+\log 2$, for large $N$ every integer $m\le N^2$ with $\omega (m)\ge 1.88L$ is exceptional there. There are only $O(N^2/L)$ such integers. Products of good pairs lie in this exceptional set; products having only bad representations are at most as numerous as the bad pairs. This proves the multiplication-table upper bound, using distinct prime factors alone:

One general form of the Erdős-Kac theorem is the following. Let $f$ be a real strongly additive arithmetic function, with $|f(p)|\le C$ for all primes, and defineIf $B(x)\to \infty$, then under the discrete uniform distribution on $[N]$,Here the arrow denotes convergence in distribution and $\mathcal{N}(0,1)$ is the standard normal distribution. Equivalently, the probability of being at most $z$ tends to the standard normal distribution function $\Phi (z)$ for every real $z$. Replacing $B(x{)}^2$ by ${\sum }_{p\le x}f(p{)}^2(1-1/p)/p$ is equivalent, because the difference is bounded by $C^2{\sum }_p{p}^{-2}$.

This is a general bounded-prime-weight form, not just the special case $f=\omega$. In that case the Mertens theorem for reciprocal primes gives $A(N)=\log \log N+O(1)$ and $B(N{)}^2=\log \log N+O(1)$, so the classical prime-factor central limit theorem is

Assume the three standard dice are fair and independent. For one die $X$, a discrete uniform distribution on $1,\dots ,6$ givesThe variance is $\mathrm{Var}X=\mathbb{E}{X}^2-(\mathbb{E}X{)}^2=35/12$. For the sum $Q=X_1+X_2+X_3$, linearity of expectation and addition of variances for independent random variables give