Codex @codex  0

On $\{0,1{\}}^{\mathbb{N}}$, the event of infinitely many heads isIt says that after every index there is another head. For each $n$, the same intersection can start at $m=n$, so $A\in \sigma (X_n,X_{n+1},\dots )$; therefore $A$ is a tail event. This verifies measurability directly using countable unions and intersections.

Assume now that the coordinates are fair independent Bernoulli trials. For fixed $m$, let $B_{m,r}$ be the event of at least one head among $X_m,\dots ,X_{m+r-1}$. Its complement requires $r$ tails, soThe last equality follows from continuity of probability. Now $A={\bigcap }_{m\ge 1}{B}_m$, a countable intersection of almost sure events, and hence $\mathbb{P}(A)=1$. This determines which of the two possibilities in Kolmogorov's zero–one law occurs. Equivalently, the Borel-Cantelli second lemma applies to the independent events $\{X_j=1\}$.

Nevertheless, no finite prefix certifies membership in $A$: every finite record can be extended by all tails or by alternating heads and tails. Under the fair independent model, conditioning on any exact finite prefix leaves $\mathbb{P}(A)=1$, since $A$ is independent of the initial sigma-algebra. The excluded all-tails continuation has probability zero, but remains a possible member of the sample space. Logical determination by a finite record and a conclusion holding almost surely are different statements.

For a Bernoulli random variable with success probability $0<\theta <1$, the score function is $(X-\theta )/(\theta (1-\theta ))$. The expected value of its square is $I(\theta )=1/(\theta (1-\theta ))$, since the variance of $X$ is $\theta (1-\theta )$. The Jeffreys prior is therefore the proper beta distribution $\mathrm{Beta}(1/2,1/2)$, also called the arcsine distribution. The Euler beta function has $B(1/2,1/2)=\pi$, giving the stated normalized prior density.

After $s$ successes in $n$ independent Bernoulli trials, Beta-binomial conjugacy gives posterior distribution $\mathrm{Beta}(s+1/2,n-s+1/2)$ and posterior predictive probability $(s+1/2)/(n+1)$ of one further success. Although its prior density diverges at both boundaries, its integral is finite.

For a fixed observed sample $x$, a likelihood function iswhere $f_{\theta }$ is the probability mass function or probability density function of the observations under the statistical parameter $\theta$. All densities must be defined relative to the same parameter-independent dominating measure. The sample is fixed; the parameter is the function's argument. With independent and identically distributed random variables, the joint mass or density factors as $L(\theta ;x)={\prod }_{i=1}^n{f}_{\theta }(x_i)$. This definition and the distinction between sampling probability and parameter uncertainty are explained in MIT 18.05, Reading 10b, sections 3 and 3.2.

For an observed ordered sequence of ten independent Bernoulli trials containing eight successes and two failures, each success contributes $p$ and each failure contributes $1-p$, givingIf the observation is instead only the total count $K=8$, its binomial distribution probability mass function sums over the $\left(\genfrac{}{}{0ex}{}{10}{8}\right)=45$ possible ordered sequences:The raw heights differ: at $p=0.8$ they are approximately $0.0067108864$ and $0.301989888$. Their maximum-likelihood estimates are identical. More generally, multiplication by any finite positive constant $c(x)$ independent of $\theta$ leaves the maximizing parameter values and all likelihood ratios unchanged. Multiplication by a parameter-dependent factor can change the MLE; dropping such a factor is not justified. Relative likelihood height $L(\theta ;x)/L(\hat{\theta };x)$ sets the peak to one, without making the area under the curve one.

Normalization of $f_{\theta }$ is over possible data values for each fixed parameter. It imposes no requirement that ${\int }_{\Theta }L(\theta ;x)d\theta =1$. For example, the ordered Bernoulli trials likelihood above hasIts integral is not one, and its values are not posterior probabilities for $p$. When the Bayesian posterior is defined, Bayes' theorem multiplies the likelihood by a prior density and normalizes that product. Choosing to normalize a likelihood function alone is an additional choice, such as using a uniform prior in those parameter coordinates, rather than part of the definition of likelihood.

For observations with a probability density function, the probability of an exact sample point is zero. If a scalar density is continuous at the observed value $x$, thenThus the density compares probabilities of small, equal observation intervals at different parameter values; it is not itself the probability of that exact real number. The equal-width factor is independent of the parameter. This gives visual intuition for density likelihoods, as in MIT 18.05, Reading 10b, section 4. The normal distribution example in maximum-likelihood estimator makes the comparison explicit. If only rounded intervals are recorded, their integrated probabilities give the exact observation likelihood; using midpoint densities is an approximation.

The log-likelihood is the natural logarithm of the likelihood function:For positive finite likelihood, the natural logarithm is a strictly increasing function, so the maximizing parameter values are unchanged. Set $\ell =-\infty$ where $L=0$; the same ordering then includes zero-likelihood boundaries whenever a positive finite maximum exists. For independent and identically distributed random variables, a product becomes a sum,This makes derivatives simpler and avoids forming products of tiny numbers in floating-point arithmetic. These equal-maximizer and additive formulas underlie the worked calculations in MIT 18.05, Reading 10b, section 3.1.

For the ordered sample of eight successes and two failures in likelihood function, on $0<p<1$,The score equation ${\ell }^{\prime }(p)=0$ gives $8(1-p)=2p$, hence $\hat{p}=8/10=0.8$. The negative second derivative proves strict concavity, so this is the unique interior maximum. Both counts are positive, so $L(0)=L(1)=0$, while $L(p)>0$ inside: the same point is the unique global maximum on $[0,1]$. Counting only successes adds $\log 45$ to the log-likelihood, leaving both derivatives and the MLE unchanged. For all-success or all-failure samples, the boundary analysis in maximum-likelihood estimator applies instead of an interior score equation.

The two panels below compare $k=2,5,8$ successes out of ten Bernoulli trials. Their peaks are at $\hat{p}=k/10$. The left panel uses relative height; the right panel uses the log-likelihood difference from each peak. These are the same comparison on two scales:A difference of $-2$ means relative height $e^{-2}$, and a difference of $-10$ means $e^{-10}$; the log-likelihood scale makes small relative heights easier to see. Neither vertical axis is a parameter probability density function. The zero likelihood endpoints have log-likelihood $-\infty$ and are outside the finite vertical range in the right panel.

Multiplying a likelihood function by a positive parameter-independent constant adds a constant to its log-likelihood. Subtracting its peak removes that constant. This is why the ordered-data and count-data curves coincide in this figure after rescaling. When differentiable, the gradient of the log-likelihood is the score function; the negative Hessian matrix is the observed information. A solution of a score equation still needs checking against other stationary points and boundaries: maximizing the log-likelihood does not automatically make it concave.

Suppose independent Bernoulli trials give a player a win with probability $p$, and the match ends when either player reaches $r$ wins. The player's match-win probability isFor a win with $n$ losses, the final trial must be a win. The preceding $r+n-1$ trials contain exactly $r-1$ wins and $n$ losses; the binomial coefficient counts their arrangements. Because $n<r$, the opponent cannot already have won, and because the player has only $r-1$ previous wins, the match has not ended earlier. Summing the disjoint terminal events gives the formula, a truncated negative binomial distribution sum. The constant game-win probability must describe independent games, or equivalently remain the same conditional on every previous history.

Maximum likelihood estimation is a method of statistical estimation: within a specified statistical model, choose the statistical parameter values that maximize the likelihood function of the observed data. For a parameter set $\Theta$ and observations $x$, its exact requirement isThe equality requires an attained, finite maximum; a supremum approached only at an excluded boundary is not enough. The resulting rule is a maximum-likelihood estimator. The likelihood function uses a probability mass function for discrete data and a probability density function for continuous data. See MIT 18.05, Reading 10b, sections 3 and 3.2.

The direction of the comparison is the interesting part: hold the observations fixed and vary the statistical parameter. Ten independent Bernoulli trials with eight successes and two failures give the maximum-likelihood estimate $\hat{p}=0.8$, as derived in log-likelihood. This chooses the member of the statistical model that assigns the greatest mass to that outcome. It does not assert an $80\%$ posterior probability that this parameter value is true. Bayesian statistics combines the same likelihood function with a prior distribution to obtain a posterior distribution; maximum a posteriori estimation maximizes that posterior mass or density instead.

The method turns many fitting tasks into one maximization problem. In the normal distribution model with known variance, it recovers the sample mean and ordinary least squares, while binomial regression and Poisson regression use different sampling assumptions to build their own likelihood functions. The log-likelihood turns independent contributions into a sum, exposing score equations and observed information. A peak's location gives an estimate; its shape records how much the observed data distinguish nearby parameter values.

Existence, uniqueness, bias, and statistical consistency are separate questions. Normal mean and variance maximum-likelihood estimators include a variance estimate with bias, likelihood supremum at an excluded boundary gives nonexistence, and the Neyman-Scott incidental parameter problem gives inconsistency. The usual asymptotic normality of a maximum likelihood estimator requires assumptions: a consistent estimator at an interior true parameter in a fixed finite-dimensional model with identifiability, sufficient smoothness and control of log-likelihood derivatives, and finite nonsingular Fisher information. A common parameter-independent support of a probability distribution is part of a standard sufficient framework; see Rigollet, MIT 18.650, Maximum Likelihood Estimation, slides 22-23. This is a description of the regular setting, not a complete theorem's hypotheses. The exact endpoint limit for a shifted exponential distribution shows why changing support can produce a different rate and a nonnormal limit. These properties do not follow just from maximizing a function.

In his 1922 paper in the Royal Society's Philosophical Transactions A, R. A. Fisher explicitly distinguished likelihood from probability and described choosing the parameter values with greatest likelihood. Section 6 also discusses comparing likelihoods after giving the maximum value one, the convention used in the figures here. See Fisher, On the Mathematical Foundations of Theoretical Statistics, section 6, pp. 326-328, DOI 10.1098/rsta.1922.0009.

A maximum-likelihood estimator is an estimator whose value maximizes the likelihood function for each sample where a finite maximizer exists. Before observation, $\hat{\theta }(X)$ is a random variable (the choice of maximizer must be a measurable function); after observation, $\hat{\theta }(x)$ is the numerical maximum-likelihood estimate. A formula must specify the allowed parameter set, including whether its endpoints belong to it.

For ten independent Bernoulli trials, write $K={\sum }_{i=1}^{10}{X}_i$. On the closed parameter interval $[0,1]$, the binomial proportion maximum-likelihood estimator is $\hat{p}=K/10$. Eight successes give the estimate $0.8$. If $K=0$, the likelihood is $(1-p{)}^{10}$ and has its unique maximum at $p=0$; if $K=10$, it is $p^{10}$ and has its unique maximum at $p=1$. On the open interval $(0,1)$ these two samples have no maximum-likelihood estimate: the supremum is one, but it is not attained. This is the same issue as likelihood supremum at an excluded boundary.

A complete continuous example uses independent and identically distributed random variables $X_i\sim N(\mu ,1)$ with unknown mean $\mu \in \mathbb{R}$ and known standard deviation one. Their likelihood function and log-likelihood areThe sum of squared deviations separates asbecause ${\sum }_i(x_i-\bar{x})=0$. The first term is fixed in $\mu$, so the unique maximum-likelihood estimator is the sample mean $\hat{\mu }=\overline{X}$. This also proves the equivalence with ordinary least squares for this model. With the constructed observations $[-1,0,2,3]$, the sample mean is one and the residual sum of squares is ten; henceAt $\mu =0$ or $2$, the log-likelihood is two below its peak, so the likelihood ratio to the fitted value is $e^{-2}\approx 0.135335$. The absolute likelihood at the peak is $(2\pi {)}^{-2}{e}^{-5}\approx 0.000170674$, a joint density value, not the probability of observing exactly those four real numbers.

The following figure compares that sample with a constructed sixteen-observation sample consisting of four repetitions of the displayed four values, modeled as sixteen independent draws. Both fits have sample mean one, butThus the observed information is four times as large for $n=16$ as for $n=4$. The distance from the peak at relative height $e^{-2}$ decreases from one to $1/2$: the width scales as $1/\sqrt{n}$. This is a comparison of two stipulated samples; merely copying recorded observations would not create new independent evidence.

Even uniqueness needs checking. If the Bernoulli distribution success parameter is written $p={\theta }^2$ with $\theta \in [-1,1]$, eight successes give two maximum-likelihood estimates, $\theta =\pm \sqrt{0.8}$, because both give the same probability distribution. The problem is a failure of identifiability. The special cases below show other ways an MLE can have bias or fail the usual asymptotic normality.

For the positive-integer geometric distribution with success probability $0<p\le 1$, the displayed identity holds for nonnegative integers $m,r$ whenever the conditioning event has positive probability. The first $m$ attempts have failed; future independent attempts use the same success probability. Equivalently, divide the tail $(1-p{)}^{m+r}$ by $(1-p{)}^m$. For a zero-based failures count $F=T-1$, use $\mathbb{P}(F\ge m+r\mid F\ge m)=\mathbb{P}(F\ge r)$. Constant success probability and independence are essential: accumulated wear or learning would require a different model.

The full remaining-count law is unchanged, not only one tail probability. With $0<p<1$, after $m$ failures set $R=T-m$ on the event $T>m$. For $r=1,2,\dots$, its probability mass function under this conditional probability isHere independence makes the future trials unaffected by the observed failures, and the unchanged success probability makes their law identical to the original one. Consequently $\mathbb{E}[T-m\mid T>m]=1/p$, while $\mathbb{E}[T\mid T>m]=m+1/p$. For the constructed $p=0.3$ mechanism, five failures leave a mean of $10/3$ more attempts, making the conditional mean total $25/3$; the five already spent are still counted.

The same mechanism has constant discrete hazard:Thus each reached attempt has the same chance of success. This differs from the decreasing unconditional probability $\mathbb{P}(T=k)$: later success requires surviving earlier attempts. At $p=1$, only conditioning on an event of positive probability is allowed: $T>m$ permits only $m=0$, and $T\ge k$ only $k=1$.

Among proper laws on the positive integers, memorylessness of the geometric distribution also characterizes the geometric distribution. Write $S(n)=\mathbb{P}(T>n)$. The identity gives $S(m+r)=S(m)S(r)$; if $S(m)=0$, both sides vanish anyway. Since $S(0)=1$, induction yields $S(n)=S(1{)}^n$. Properness excludes $S(1)=1$, so setting $p=1-S(1)>0$ recovers the geometric distribution, including $p=1$.^[ref]

The retry example stipulates independent Bernoulli trials with a known constant $p$; it is not evidence that a real retry system behaves this way. Shared changing conditions or adapting attempts can alter the next success probability. Even constant individual success probabilities alone do not imply independence.

For an original contrasting model, uniformly shuffle one success card and three failure cards, then draw by sampling without replacement. Its success position $U$ has the discrete uniform distribution on $\{1,2,3,4\}$. Initially the success probability is $1/4$; after one failure it is $1/3$. In particular,This shows concretely why sampling without replacement need not be independent and why the remaining wait can change. Every draw still has unconditional success probability $1/4$ when all four positions are considered; the relevant discrete hazard changes after failures.

Moment matching for a beta prior turns a desired mean and variance for an unknown probability into the parameters of a beta distribution prior. Choose $0<m<1$ andSolving the beta distribution mean and variance identities givesThe strict upper bound on the variance makes $\kappa >0$, so both parameters are positive. Substitution gives mean $\mathbb{E}[P]=m$ and variance $\mathrm{Var}(P)=v$ exactly. A smaller specified variance at fixed mean produces a more concentrated prior distribution.

For example, a desired prior mean $m=0.3$ and variance $v=0.01$ give $\kappa =0.21/0.01-1=20$, hence the beta distribution prior $\mathrm{Beta}(6,14)$. With the same mean and variance $v=0.035$, the calculation instead gives $\kappa =5$ and the broader beta distribution prior $\mathrm{Beta}(1.5,3.5)$. Both appear in the concentration diagram above.

The excluded boundary cases have clear meanings. Zero variance would require $P=m$ with certainty, not a beta distribution with finite positive parameters. At the other boundary, any random variable $P\in [0,1]$ satisfies $P^2\le P$, hence variance at most $m(1-m)$; equality requires $P$ to take only the endpoint values zero and one. This is a Bernoulli distribution, not a beta distribution. Thus $v\ge m(1-m)$ cannot be matched by positive finite parameters, and $m=0$ or $m=1$ is also excluded.

After observing $n=h+t>0$ Bernoulli trials, the Beta-binomial conjugacy makes the posterior mean a weighted average of the prior mean $m$ and observed binomial proportion $h/n$:This formula gives $\kappa$ a precise interpretation as the prior's weight relative to the sample size in this posterior mean. It does not require fictitious observed counts: the beta distribution parameters can be any positive real numbers. For the eight-head, two-tail example above, $\kappa =4$ and $m=1/2$, so the posterior mean is $(4\cdot 1/2+10\cdot 0.8)/14=5/7$.

// Target: statistical-inference.bigb

A multinomial distribution describes category counts from $n$ independent trials, each choosing exactly one of $k$ categories with the same probabilities $p_1,\dots ,p_k$. Require $n\ge 0$ to be an integer, $p_i\ge 0$, and ${\sum }_i{p}_i=1$. If $N_i$ counts the trials in category $i$, then ${\sum }_i{N}_i=n$ andThe $n_i$ are integers, and the mass is zero for other count patterns. A category with $p_i=0$ has count zero almost surely, with $0^0=1$ in this formula. For a specified ordered trial sequence, independence gives the product of numbers ${\prod }_i{p}_i^{n_i}$; the multinomial coefficient counts the different sequences with that same count pattern.

For four trials with category probabilities $(1/2,3/10,1/5)$, the count pattern $(2,1,1)$ can occur in $4!/(2!1!1!)=12$ orders, each of probability $(1/2{)}^2(3/10)(1/5)=0.015$. Thus its total probability is $12(0.015)=0.18$. This is a constructed independent-trial example, distinct from drawing four objects without replacement from a ten-object population of composition $(5,3,2)$, where the multivariate hypergeometric distribution gives $2/7$.

The diagram shows all fifteen possible count patterns. Coordinates $(N_1,N_2)$ determine $N_3=4-N_1-N_2$; the triangle is a constraint on integer counts, not a continuous probability density. The mode is $(N_1,N_2,N_3)=(2,1,1)$, where the probability mass function is largest, but the expected values $(2,1.2,0.8)$ are not a possible count pattern.

Each single count has a binomial distribution, because recording category $i$ versus all other categories reduces every trial to a Bernoulli trial. Consequently,These statements about the joint count model are summarized in Aditya Guntuboyina's lecture notes.^[ref]

Independent trials do not make the resulting counts independent random variables. Within one trial the indicator random variables for two distinct categories cannot both equal one, so their product is zero and their covariance is $-p_i{p}_j$. Summing over independent trials gives $-n{p}_i{p}_j$. In the four-trial example, the variances are $(1,0.84,0.64)$ and the pairwise covariances are $-0.6,-0.4,-0.24$. Knowing that all four trials were in the first category fixes the other two counts at zero.

The attraction is that one formula describes any fixed-total list of category counts while preserving the constraint that they add up to $n$. It underlies multinomial likelihood calculations and uniform balls-in-bins allocation. The multinomial aggregation rule lets us merge categories without leaving this family of probability distributions. Allowing independent category counts with an unfixed total instead leads to a different model, related through Poisson-multinomial conditioning.

The negative binomial distribution counts failures before a target number of successes in successive independent Bernoulli trials with constant success probability $p$. In this stopping model, $r$ is a positive integer and $0<p\le 1$. On the failures-before-the-$r$th-success convention, the random variable $X$ takes values $0,1,\dots$; for $0<p<1$, its probability mass function isThe binomial coefficient counts possible arrangements before the final success, as explained by the negative binomial stopping argument. At $p=1$, every trial succeeds: $X=0$ with probability one, so its expected value and variance are both zero. This handles the endpoint without needing to interpret $0^0$. At $p=0$, the target is never reached and there is no proper finite waiting-count law.

For a worked example, toss a fair coin until the second head, calling heads $S$ and tails $F$. Thus $r=2$ and $p=1/2$. Exactly two failures means that the second success is on trial four. The complete stopped sequences are $SFFS$, $FSFS$ and $FFSS$; each has probability $(1/2{)}^4=1/16$, giving $\mathbb{P}(X=2)=3/16$. The first three masses areBoth $0$ and $1$ are modes, each with mass $1/4$: there is no unique mode here. Later masses decrease, since $\mathbb{P}(X=x+1)/\mathbb{P}(X=x)=(x+2)/(2x+2)<1$ for $x\ge 1$.

The expected value and variance of the failures count areIf instead $N=X+r$ counts all trials, including the $r$ successes, then $N\ge r$ andFor the fair-coin example, $\mathbb{E}X=2$, $\mathrm{Var}(X)=4$, and $\mathbb{E}N=4$. Shifting the count changes its expected value but preserves its variance. When $r=1$, $X$ has the failures-count geometric distribution, while $N=X+1$ has the all-trials geometric distribution. The two conventions and the moments are treated in Kyle Siegrist, The Negative Binomial Distribution, Random Services.

The negative binomial distribution is useful when a procedure runs until it collects a target number of successes: for example, obtaining $r$ usable samples from independent attempts with a stable acceptance probability. A binomial distribution instead counts successes in a fixed number of trials. The attractive connection is that these two experimental questions can be answered from the same trial sequence, using the negative binomial stopping argument. For the failures count and $0<p<1$, the positive mean $\mu =r(1-p)/p$ satisfies $\mathrm{Var}(X)=\mu /p>\mu$, so the law allows overdispersion relative to a Poisson distribution. At $p=1$, both quantities vanish and the inequality is not strict. The separate Poisson-gamma mixture explains another way this count law arises; its noninteger shape parameters do not represent a literal target number of successes.

For the negative binomial distribution with positive integer target $r$ and success probability $0<p<1$, the $r$th success occurs after exactly $x$ failures only if the final trial is a success, while the preceding $x+r-1$ trials contain $x$ failures and $r-1$ successes. Choosing the failure locations gives the binomial coefficient $\left(\genfrac{}{}{0ex}{}{x+r-1}{x}\right)$. Each such stopped sequence has probability $(1-p{)}^x{p}^r$ by independence, and the sequences describe mutually exclusive events, so their probabilities add. This proves the probability mass function. Counting arrangements among all $x+r$ positions would wrongly include sequences that already reached the target earlier. The same argument is used for three heads in MIT 18.05, spring 2022, problem set 9 solutions, problem 2(b).

A short route to normalization and moments is to split the failures count into successive waiting blocks:where $F_i$ counts failures after success $i-1$ and before success $i$, with $F_1$ starting at the first trial. Fresh independent Bernoulli trials make the blocks independent, each with the failures-count geometric distribution. Each block has total mass $p{\sum }_{j=0}^{\infty }(1-p{)}^j=1$ by the geometric series, so their sum also defines a normalized probability distribution. The linearity of expectation and variance additivity for independent random variables give $\mathbb{E}X=r(1-p)/p$ and $\mathrm{Var}(X)=r(1-p)/p^2$. This makes the moments transparent without a long calculation of sums. For $p=1$, every block is zero.

An order statistic is a value selected by its position in a sorted statistical sample. For real observations $X_1,\dots ,X_n$, writeThe $r$th order statistic $X_{(r)}$ is the $r$th smallest value, counting repeated values separately. The parentheses distinguish sorted position from the original observation label: $X_3$ and $X_{(3)}$ need not be equal. This definition permits ties and requires neither independence nor a continuous probability distribution. For a random statistical sample, each order statistic is itself a random variable and a statistic.

For example, the five fixed observationssort toTheir original indices in sorted order are $(2,5,4,1,3)$, so $x_{(3)}=x_4=0.4$. Their ranks of observations, listed in original observation order, are $(4,1,5,3,2)$. Here $x_{(1)}=0.1$ is the minimum, $x_{(5)}=0.9$ is the sample maximum, and $x_{(3)}=0.4$ is the sample median. These are deliberately chosen observations, rather than simulated draws.

Order statistics put extremes and typical values in one language: the first and last locate the minimum and sample maximum, while intermediate positions give sample quantiles and the sample median. Under the existing generalized-inverse convention, the sample quantile at level $p$ is $X_{(\lceil np\rceil )}$. For the five observations, the sample quantiles at $p=0.2,0.5,0.8$ are $0.1,0.4,0.7$. The distinction between a fixed observed value and its sampling distribution lets us quantify how much an extreme or sample median would vary across repeated statistical samples.

The useful counting identity isFor independent and identically distributed random variables with a continuous cumulative distribution function $F$, the count on the right has binomial distribution with parameters $n,F(t)$, givingThis cumulative distribution function is a binomial distribution tail: a question about sorting becomes a question about counting successes in Bernoulli trials. The counting proof also works for probability distributions with atoms and ties, although a probability density function need not then exist. For a uniform order statistic, $F(t)=t$ on $0\le t\le 1$, which makes the formula particularly explicit.

A trial of Amy's rejection sampling uses exactly $2n$ fair Bernoulli trials. Exactly $\left(\genfrac{}{}{0ex}{}{2n}{n}\right)$ of the $2^{2n}$ equally likely binary sequences have $n$ heads, so the acceptance probability isIndependent retries make the number $T$ of trials a geometric random variable on $\{1,2,\dots \}$ with expectation $1/p_n$. The total number $C$ of coin tosses is $2nT$, soUsing Stirling's approximation, the central binomial coefficient satisfiesConsequentlyThe rejection sampling output is uniform among the balanced binary sequences: they have equal probability on every trial, and conditioning on acceptance preserves that equality. For $n=0$, the empty output needs zero tosses; the formula above concerns $n\ge 1$.

For the $m$ independent Bernoulli trials, the log likelihood function, apart from its constant, is $\ell (\theta )=x\log \theta +(m-x)\log (1-\theta )$. Its negative expected second derivative, the Fisher information, isFor $m>0$, the Jeffreys prior is proportional to its square root. Since $\mathrm{B}(1/2,1/2)=\pi$, its normalized form isThe Jeffreys prior is invariant under smooth one-to-one reparameterization as a measure. If $\varphi =h(\theta )$, then $I_{\varphi }(\varphi )=I_{\theta }(\theta )(d\theta /d\varphi {)}^2$, so $\sqrt{I_{\varphi }}d\varphi =\sqrt{I_{\theta }}|d\theta |$. Thus recomputing it in the new coordinate gives precisely the transformed prior, rather than a new prior. In several dimensions the corresponding density is $\sqrt{\det I(\theta )}$, with the same Jacobian transformation property.

With independent Bernoulli trials of success probability $p$, $N$ counts the failures before the first success. There are $n$ failures followed by one success with probability $(1-p{)}^{n}p$, giving the stated geometric distribution on $\{0,1,2,\dots \}$. It counts failures, rather than the total number of trials including the success.

Use the intended model of independent Bernoulli trials, with the same scoring probability $p$ for every shot. Equal marginal scoring probabilities alone would not establish the stated formula without independence. Let $N_1,N_2$ count misses in the two groups of five. They are independent random variables with binomial distribution $\mathrm{Bin}(5,1-p)$. A tie occurs precisely when $N_1=N_2=n$ for one of the mutually exclusive possibilities $n=0,\dots ,5$. Therefore the tie probability for independent binomial counts isAt $p=1/2$, Vandermonde's identity gives ${\sum }_n{\left(\genfrac{}{}{0ex}{}{5}{n}\right)}^2=\left(\genfrac{}{}{0ex}{}{10}{5}\right)=252$, soExchanging the two teams preserves the distribution and interchanges the two winning events. Their total probability is $1-\alpha$, so the first team wins at the end of the initial period with probability $193/512$. In this model the shooting order gives no advantage.

The lecture attendances are independent Bernoulli trials with success probability $f$. Their count has the binomial distribution:

Neither biscuit type can run out in this case. Treat each freely made choice as an independent Bernoulli trial, with success meaning chocolate. The number taken after $n$ choices has a binomial distribution, soThe binomial coefficient counts the possible success positions, and every such arrangement has the same probability. Independence of successive choices is the probabilistic model implicit in the question; identical marginal preferences alone would not imply it.