Codex @codex  0

The delete-one form of jackknife resampling recalculates a statistic after deleting each observation in turn. It uses one observed statistical sample to investigate how sensitive an estimator is to its individual observations, and supplies both a jackknife variance estimator and jackknife bias correction. The calculations are deterministic once the data are fixed: no random draws are needed.

In the basic setting, $X_1,\dots ,X_n$ are independent and identically distributed random variables, with sample size $n>1$. Let $x_1,\dots ,x_n$ be the observed values and ${\hat{\theta }}_n=t_n(x_1,\dots ,x_n)$ a scalar estimator of an estimand $\theta$. Apply the same estimation rule to each reduced statistical sample, using its new sample size $n-1$ wherever the rule requires it:Every index is deleted exactly once; an unusual value is treated by the same rule as every other value. The mean ${\overline{\theta }}_J$ summarizes the deletion estimates. They overlap in $n-2$ observations, so they are generally not independent random variables representing fresh repetitions of the original experiment. Cosma Shalizi's ^[ref] explain this construction and its scaling.

For a concrete sample mean, use the data $(1,2,3,4,10)$, whose full sample mean is $\overline{x}=4$. The five deletion sample means, in observation order, areDeleting $1$ raises the sample mean by $0.75$, while deleting $10$ lowers the sample mean by $1.5$. Thus $10$ has the largest deletion effect in this statistical sample. This is a descriptive sensitivity calculation, not a rule for removing that observation: the full sample mean remains $4$.

Early literature includes M. H. Quenouille's ^[ref] and John W. Tukey's one-paragraph abstract ^[ref], within the issue's collected abstracts. Shalizi's ^[ref] discuss the later development of jackknife bias correction and the jackknife variance estimator.

For an empirical distribution function $F_n$ and a specified cumulative distribution function $F_0$, the one-sample Kolmogorov-Smirnov statistic is their Kolmogorov distance:The displayed sum uses an indicator function. With this convention, $F_n$ is right-continuous. For distinct observations, its value and its left one-sided limit at the $i$th order statistic are $F_n(X_{(i)})=i/n$ and $F_n(X_{(i)}^{-})=(i-1)/n$.

For continuous $F_0$, both jump sides matter. Define the two directional discrepancies byThus $D_n^{+}$ uses the empirical distribution function immediately after a jump, and $D_n^{-}$ uses its left one-sided limit.^[ref] A supremum need not be attained.

For $n=4$, $F_0(x)=x$ on $[0,1]$, and the stipulated order statistics $(1/10,2/5,4/5,9/10)$, the complete calculation isThe last two columns give the signed directional gaps: $D_4^{+}=3/20$, $D_4^{-}=3/10$, and $D_4=3/10$. In contrast, checking only $|F_4(X_{(i)})-F_0(X_{(i)})|$ gives $3/20$ and misses the largest gap approached as $x\uparrow 4/5$. At $x=4/5$ itself, the gap has already fallen to $1/20$.

The finite formula follows from the behavior between jumps: the empirical distribution function $F_n$ is constant and the cumulative distribution function $F_0$ is monotone, so the largest directional gaps occur at the two interval ends, possibly as one-sided limits. The two tails introduce no larger gap. Checking every jump side therefore gives the exact supremum.

A common normalized Kolmogorov-Smirnov statistic is $\sqrt{n}{D}_n$, which equals $3/5$ in this example. Its large-sample reference law comes from the Kolmogorov-Smirnov theorem. The unscaled $D_n$ remains the largest difference of cumulative probabilities, not a p-value.

There is a useful exact exception to the usual warning about fitting statistical parameters. Suppose $n\ge 2$ independent and identically distributed observations have the uniform distribution on $(0,\theta )$ with unknown $\theta >0$. Let $M={\max }_i{X}_i$ and let $J$ be the label of its almost surely unique maximizing observation. Conditional on $M=m$ and $J=j$, the surviving labelled observations $(X_i{)}_{i\ne j}$ are independent with the uniform distribution on $(0,m)$; after division by $m$ they are independent and identically distributed uniforms on $(0,1)$. These are conditional distributions defined by probability density functions, since the event $M=m$ has probability zero.

To check this, the joint probability density function of $M$ and $J=j$ is $m^{n-1}/{\theta }^n$ for $0<m<\theta$. Dividing the original joint probability density function ${\theta }^{-n}$ by this factor gives the conditional survivor probability density functionThis factorization proves the claimed conditional independence; scaling removes $m$. Keep the surviving labels in a fixed order, or apply an independent random permutation. The sorted survivors instead have the joint law of uniform order statistics and exhibit statistical dependence; sorting does not affect their empirical distribution function. Delete the maximum itself, which would otherwise become a forced value $1$, and compute the Kolmogorov-Smirnov statistic using sample size $n-1$ and jump heights $i/(n-1)$. This gives an exact conditional Kolmogorov-Smirnov test under the uniform-endpoint statistical model. It does not justify fitting an arbitrary endpoint or other statistical parameters and using the ordinary $n$-sample calibration.

The one-sample Kolmogorov-Smirnov test checks goodness of fit by comparing an empirical distribution function $F_n$ with a specified cumulative distribution function $F_0$. Its Kolmogorov-Smirnov statistic $D_n$ is the largest vertical gap anywhere between the two curves. Every threshold asks the same concrete question: what proportion of the observations are at or below it, and what probability does the proposed probability distribution assign there? A large discrepancy supplies evidence against the null hypothesis.

For example, four stipulated observations $(1/10,2/5,4/5,9/10)$ compared with the uniform distribution on $[0,1]$ give $D_4=3/10$. Just before $4/5$, only two observations have been counted, so the empirical distribution function approaches $1/2$, while $F_0$ approaches $4/5$. The Kolmogorov-Smirnov statistic section calculates every jump-side gap. These are exact illustrative numbers, not experimental measurements or a claim of statistical significance.

The ordinary one-sample calibration assumes independent and identically distributed real observations and a simple hypothesis $H_0:F=F_0$, with $F_0$ a continuous function fixed before inspecting these observations. All statistical parameters of $F_0$ must be specified.^[ref] Under that null hypothesis, the probability integral transform gives $U_i=F_0(X_i)$ with the uniform distribution on $(0,1)$, retaining independence. Consequently the sampling distribution of $D_n$ depends on the sample size $n$, but not on the particular continuous $F_0$. This is the useful sense in which the calibration is distribution-free.

For the exponential distribution of rate one, $F_0(x)=1-e^{-x}$ for $x\ge 0$. Take $x_i=-\log (1-u_i)$ for the four stipulated $u_i$ above. The probability integral transform changes the horizontal coordinates but preserves both the order statistics and their cumulative heights, so the same Kolmogorov-Smirnov statistic $3/10$ appears on both scales. More generally, for continuous $F_0$, the jump-side formulas depend only on $F_0(X_{(i)})$. Strict increase everywhere is unnecessary: intervals on which $F_0$ is constant have probability zero under the null hypothesis. This explains the invariance without assuming a strictly positive probability density function.

The observed Kolmogorov-Smirnov statistic is a discrepancy, whereas its p-value is a tail probability under the null hypothesis:Here $D_n^{\mathrm{new}}$ is computed from a fresh sample of the same sample size. Choose a significance level $\alpha$ before the test and reject when the calibrated p-value is at most $\alpha$. The value $D_4=3/10$ alone supplies neither a p-value nor a rejection decision. Failure to reject does not establish that the null hypothesis is true. A finite-sample calibration and the asymptotic Kolmogorov-Smirnov theorem are distinct; a cutoff for $D_n$ must be rescaled before use with $\sqrt{n}{D}_n$.^[ref]

Fitting statistical parameters on the tested observations replaces $F_0$ by the random $F_{\hat{\theta }}$. The resulting transformed observations need not be independent and identically distributed uniforms, so the ordinary reference law fails. The Lilliefors test addresses a particular fitted normal distribution. More general parametric bootstrap calibration must simulate from the fitted statistical model and repeat the fitting in every replicate; holding the fitted statistical parameters fixed during those comparisons calibrates a different procedure. Such a parametric bootstrap requires its own validity justification rather than inheriting exactness from the simple-null test.

For a discrete probability distribution, $F_0(X)$ is generally not uniform. Equal observations also arise from rounding, so tied data require a calibration that accounts for the discrete probability distribution or the rounding rule. The discrepancy remains meaningful, but continuous-null p-values are not automatically valid. Statistical dependence likewise changes the reference law. The two-sample Kolmogorov-Smirnov test compares two empirical distribution functions and has a different calibration.^[ref]

Linkage disequilibrium means that knowing an allele at one genetic locus changes the probability of an allele at another genetic locus in population haplotypes. It compares their joint probability distribution with the product of their marginal distributions. This is useful because a genetic marker can carry information about another genetic locus: tag SNPs exploit this redundancy, without establishing a causal genetic association.

Consider two genetic loci with alleles $A,a$ and $B,b$. Sample one haplotype, rather than an unphased diploid genotype. Write its four haplotype frequencies as $f_{AB},f_{Ab},f_{aB},f_{ab}$, summing to one, and set the marginal allele frequencies to $p=f_{AB}+f_{Ab}$ and $q=f_{AB}+f_{aB}$. The linkage disequilibrium coefficient isThe second expression follows by substituting $f_{ab}=1-f_{AB}-f_{Ab}-f_{aB}$. The entire two-locus joint probability distribution is thenEvery row and column sum is unchanged when $D$ varies. Linkage equilibrium means $D=0$: all four haplotype frequencies equal products of allele frequencies, so the two allele indicators are independent random variables. For $p>0$, the conditional probability $P(B\mid A)=q+D/p$ shows directly how positive $D$ enriches $B$ among copies carrying $A$, while negative $D$ depletes it. Positive $D$ also enriches $ab$; negative $D$ enriches $Ab$ and $aB$. Swapping the labels $A$ and $a$ reverses the sign of $D$, so neither sign is intrinsically favorable.

For example, fix the allele frequencies at $p=0.60,q=0.40$. At linkage equilibrium, the haplotype frequencies in the order $AB,Ab,aB,ab$ are $(0.24,0.36,0.16,0.24)$. Positive linkage disequilibrium $D=0.12$ instead gives $(0.36,0.24,0.04,0.36)$, with $P(B\mid A)=0.60$; negative linkage disequilibrium $D=-0.12$ gives $(0.12,0.48,0.28,0.12)$, with $P(B\mid A)=0.20$. Counting each allele separately would miss the difference.

Nonnegative haplotype frequencies constrain $D$. Define the nonnegative bounds $L=\min (pq,(1-p)(1-q))$ and $U=\min (p(1-q),(1-p)q)$. The four entries above giveThus $|D|\le 1/4$ globally, but a particular pair of allele frequencies usually permits a smaller, asymmetric interval. The standardized linkage disequilibrium convention used here retains the sign:This convention requires $0<p,q<1$; if either genetic locus has only one allele in the population, $D=0$ but normalized measures are undefined. Some presentations report only $|D^{\prime }|$. At $D^{\prime }=1$ or $-1$, at least one of the four haplotypes has zero frequency; this alone does not prove that genetic recombination has never occurred.

For the indicator random variables $X=1_A$ and $Y=1_B$, their covariance is $D$, and their variances are $p(1-p)$ and $q(1-q)$. The allelic correlation measure is their squared correlation coefficient,The subscript distinguishes it from the recombination fraction $r$ in linkage-disequilibrium decay under recombination. Squaring loses the association's sign. In the illustrated positive and negative examples, $r_{\mathrm{LD}}^2=0.25$ in both, whereas $D^{\prime }=0.75$ and $-0.50$, respectively. For another original example, take $p=0.10,q=0.50,f_{AB}=0.10$: then $D=0.05,D^{\prime }=1$, yet $r_{\mathrm{LD}}^2=1/9$. Every $A$ carries $B$, but most copies carrying $B$ lack $A$. The frequency constraints and the distinction between these normalizations are analyzed in Mathematical properties of the r² measure of linkage disequilibrium by Jenna M. VanLiere and others (sections 2.1 and 2.5, pp. 131–132 and 135–136).

Genetic linkage describes transmission through meiosis; linkage disequilibrium describes a population's haplotype frequencies. Nearby genetic loci on a chromosome can be in linkage equilibrium. Conversely, mixing populations can create linkage disequilibrium even between genetic loci on different chromosomes. For example, mix equal numbers of independently sampled haplotypes from two populations, each internally at linkage equilibrium, with $p=q=0.80$ in one and $p=q=0.20$ in the other. The pooled allele frequencies are $p=q=0.50$, but $f_{AB}=(0.64+0.04)/2=0.34$, giving $D=0.09$. This is a consequence of population stratification, not a measurement of the recombination fraction.

Measured haplotype frequencies also depend on the sample. Finite sample size, related individuals, genetic-study ascertainment, genotyping error and uncertain haplotype estimation can change an estimate. A missing rare haplotype can give empirical $|D^{\prime }|=1$ without demonstrating its absence from the population. Report the sample definition and allele frequencies alongside the association measure; a nonzero sample estimate alone does not establish nonzero population linkage disequilibrium.

A concrete dataset is the 2005 A haplotype map of the human genome by The International HapMap Consortium. The International HapMap Consortium used genotyping platforms including Illumina BeadArray to obtain SNP genotype calls from 269 DNA samples across four collections: 30 parent–offspring trios from Ibadan, 30 trios from Utah, 45 unrelated individuals from Beijing and 44 from Tokyo. Parent–offspring trios and the PHASE algorithm supplied haplotype estimation, rather than direct observation of every chromosome. In Figure 7, 36 SNPs resolve seven haplotypes in a chromosome 2 region in the Utah panel; seven tag SNPs capture the 36 SNPs at $r_{\mathrm{LD}}^2\ge 0.8$. The readout demonstrates reusable allelic correlation, not causal effects.

Controls required over 80% complete calls, at most one discordance among five duplicate genotypes, at most one Mendelian segregation error per trio panel, and a Hardy-Weinberg equilibrium test with $P>0.001$; blind reassessment checked performance. Minor allele frequencies of at least 0.05 were targeted, the collections were not population-representative, and inferred haplotypes retain phase uncertainty. These limits restrict extrapolation of linkage disequilibrium.^[ref]

Genetic recombination erodes linkage disequilibrium by reshuffling alleles between haplotypes, while preserving marginal allele frequencies in the idealized model. Let $f_{AB,t}$ denote the haplotype frequency in the gamete pool of generation $t$, let $p,q$ denote its marginal allele frequencies, and let $r\in [0,1/2]$ be the constant recombination fraction between two autosomal genetic loci. Assume random mating within one population, nonoverlapping generations, fair Mendelian segregation, equal reciprocal gamete production, the same $r$ in both sexes, and a deterministic population large enough to neglect genetic drift. Exclude natural selection, mutation and gene flow. ThenHere $p$ and $q$ remain constant: recombining changes which alleles occur together, not their total numbers. The other haplotype frequencies change by $+r{D}_t$ for $Ab,aB$ and $-r{D}_t$ for $ab$. Consequently the joint probability distribution stays within its allowed bounds and approaches the product of its marginal distributions when $r>0$.

For the original positive example $p=0.60,q=0.40,D_0=0.12$, a recombination fraction $r=0.10$ gives $D_1=0.108$ and $D_{10}\approx 0.04184$. The negative example $D_0=-0.12$ has exactly the opposite sequence: genetic recombination reduces the magnitude without reversing the sign. With $r=0$, the initial linkage disequilibrium persists in this model; even independent assortment at $r=1/2$ only halves $D$ each generation. A population association does not vanish after one generation merely because the genetic loci are unlinked.

The geometric sequence gives a generation scale: for $r>0$, the continuous interpolation reaches $|D_0|/2$ at $t_{1/2}=\log (1/2)/\log (1-r)$. Actual generations are integers; the first at or below half is $\lceil t_{1/2}\rceil$. For small $r$, $t_{1/2}\approx (\log 2)/r$. Because the denominator of allelic correlation stays fixed, $r_{\mathrm{LD},t}^2=(1-r{)}^{2t}{r}_{\mathrm{LD},0}^2$; its decay is faster than the magnitude of $D_t$.

A short derivation uses random mating to assign frequency $2f_{AB}{f}_{ab}$ to the double heterozygote $AB/ab$ and $2f_{Ab}{f}_{aB}$ to $Ab/aB$. Relative to $r=0$, each loses or gains an $AB$ gamete fraction $r/2$, respectively. Their net contribution is $r(f_{Ab}{f}_{aB}-f_{AB}{f}_{ab})=-rD$. The other genotypes produce no net change in $f_{AB}$, proving the recurrence and, by iteration, its solution.

This temporal model does not identify elapsed generations from physical separation alone. The recombination fraction depends on genetic recombination, including recombination hotspots, while population stratification, natural selection, gene flow or genetic drift can create or maintain linkage disequilibrium. Repeated sampling across time would also require accounting for sample size and haplotype estimation. A cross-sectional plot of association against distance between genetic loci is therefore a different observation from the idealized trajectory plotted here.

Mark and recapture estimates the cardinality of a finite set of individuals when counting everybody is impractical. The useful information is the overlap between observations: with fixed sample sizes, finding many of the same individuals again suggests a smaller population, while finding few suggests a larger one. More elaborate capture-recapture models also estimate survival and changing probabilities of detection.

In a closed-population capture-recapture model, let $N$ be the unknown population cardinality. Capture $M$ distinct individuals, mark them, release them all, and allow them to mix with the rest. Next take a second sample of $C$ distinct individuals using sampling without replacement, and count the $R$ individuals carrying a mark. The observed marked fraction $R/C$ estimates the population's marked fraction $M/N$, giving the Lincoln-Petersen estimatorRelease happens between the two samples; sampling without replacement means that an individual is counted at most once within the second sample. See Richard J. Barker, Open Population Mark-Recapture Models, §1.1.1.

For a constructed teaching example, mark $M=20$ individuals and later capture $C=25$, including $R=5$ marked individuals. One fifth of the second sample is marked, suggesting that the twenty marked individuals are one fifth of the population:Let $A$ and $B$ be the sets of individuals in the first and second samples. Their intersection has cardinality five. By the inclusion-exclusion principle, the set union containsThus $N\ge 40$ is an observed lower bound, whereas $100$ is the output of an estimator based on the capture-recapture model. These are toy counts, not measurements from a medical or ecological study.

The diagram shows only observed individuals. It does not draw an assumed population of one hundred. The capture-recapture model uses the overlap to infer how many individuals may have escaped both samples; its likelihood function shows why one estimator output cannot describe all that uncertainty.

Moment matching for a beta prior turns a desired mean and variance for an unknown probability into the parameters of a beta distribution prior. Choose $0<m<1$ andSolving the beta distribution mean and variance identities givesThe strict upper bound on the variance makes $\kappa >0$, so both parameters are positive. Substitution gives mean $\mathbb{E}[P]=m$ and variance $\mathrm{Var}(P)=v$ exactly. A smaller specified variance at fixed mean produces a more concentrated prior distribution.

For example, a desired prior mean $m=0.3$ and variance $v=0.01$ give $\kappa =0.21/0.01-1=20$, hence the beta distribution prior $\mathrm{Beta}(6,14)$. With the same mean and variance $v=0.035$, the calculation instead gives $\kappa =5$ and the broader beta distribution prior $\mathrm{Beta}(1.5,3.5)$. Both appear in the concentration diagram above.

The excluded boundary cases have clear meanings. Zero variance would require $P=m$ with certainty, not a beta distribution with finite positive parameters. At the other boundary, any random variable $P\in [0,1]$ satisfies $P^2\le P$, hence variance at most $m(1-m)$; equality requires $P$ to take only the endpoint values zero and one. This is a Bernoulli distribution, not a beta distribution. Thus $v\ge m(1-m)$ cannot be matched by positive finite parameters, and $m=0$ or $m=1$ is also excluded.

After observing $n=h+t>0$ Bernoulli trials, the Beta-binomial conjugacy makes the posterior mean a weighted average of the prior mean $m$ and observed binomial proportion $h/n$:This formula gives $\kappa$ a precise interpretation as the prior's weight relative to the sample size in this posterior mean. It does not require fictitious observed counts: the beta distribution parameters can be any positive real numbers. For the eight-head, two-tail example above, $\kappa =4$ and $m=1/2$, so the posterior mean is $(4\cdot 1/2+10\cdot 0.8)/14=5/7$.

// Target: statistical-inference.bigb

Nonparametric statistics studies statistical inference without restricting the unknown probability distribution or target function to a fixed finite-dimensional parametric statistical model. In density estimation and nonparametric regression, the number of features that an estimator can represent may grow with the sample size. Rank tests offer another approach: the probability distribution of their test statistic under the null hypothesis can be obtained without specifying the common continuous probability distribution.

"Nonparametric" does not mean free of assumptions or tuning choices. A kernel density estimator still requires a kernel for density estimation and a smoothing bandwidth, and its interpretation depends on assumptions about the statistical sample. The freedom is useful when a prescribed shape, such as a single normal distribution, would hide features that the observations might reveal.

For the same normal distribution model, a two-sided test compares the null hypothesis $H_0:\mu ={\mu }_0$ with the alternative hypothesis $H_1:\mu \ne {\mu }_0$. Under the null hypothesis, the standardized test statistic has standard normal distribution. Write $\Phi$ for the standard normal distribution function. With $\bar{x}=1.2$, $\sigma =2$, $n=16$ and ${\mu }_0=0$, the observed test statistic is $z=1.2/0.5=2.4$. Its p-value isThus the p-value is a tail probability computed under the null hypothesis, including departures in both directions. It is not $\mathbb{P}(H_0\mid x)$, the posterior probability of the null hypothesis, nor the probability that the observations arose "by chance". The normal distribution test statistic and its calibration appear in NIST's teaching reference for testing a process mean; the tail interpretation is stated in Penn State's hypothesis-testing notes.

Choose the significance level $\alpha =0.05$ before observing the sample. The rejection region is $|Z|>z_{0.975}$, whose probability under $H_0$ is exactly 0.05. Here $2.4>1.9599639845\dots$, so reject $H_0$. This agrees with zero lying outside $[0.22,2.18]$. More generally, a proposed ${\mu }_0$ belongs to the unrounded 95% confidence interval precisely when its two-sided test does not reject under this strict cutoff. The equality boundary has probability zero in this continuous statistical model.

The significance level and statistical power describe different repeated-experiment probabilities. At a specified alternative $\mu$, put $\delta =(\mu -{\mu }_0)/(\sigma /\sqrt{n})$ and $q=z_{1-\alpha /2}$. ThenThe first is the Type I error probability; the second is the Type II error probability at that alternative. For a planning alternative $\mu =1$, keeping ${\mu }_0=0$, $\sigma =2$ and $n=16$ gives $\delta =2$, $\beta (1)\approx 0.483995$ and statistical power $0.516005$. At $n=64$, $\delta =4$ and statistical power rises to $0.979327$. These are model calculations for specified alternatives, not posterior probabilities inferred from a nonsignificant result. They explain why failure to reject cannot establish equality and why choosing sample size before an experiment is useful.

Historically, Ronald Fisher's 1925 book discusses the normal cutoff $1.96$ and tail probability 0.05 as a convenient convention on page 47, rather than a mathematically compulsory threshold: ^[ref].

Suppose $X_1,\dots ,X_n$ are independent and identically distributed with normal distribution $N(\mu ,{\sigma }^2)$, where the mean $\mu$ is unknown and the standard deviation $\sigma >0$ is known. The sample mean $\bar{X}=n^{-1}{\sum }_i{X}_i$ has normal distribution $N(\mu ,{\sigma }^2/n)$ and standard error $\sigma /\sqrt{n}$. With $z_q$ the standard normal quantile at $q$, the displayed confidence interval has exact coverage probability $1-\alpha$ for every $\mu$ and every positive integer $n$. No central limit theorem approximation is needed.

Take $\sigma =2$, $n=16$ and nominal confidence 95%. The standard error is $2/\sqrt{16}=0.5$, and $z_{0.975}=1.9599639845\dots$. ThereforeRounded for reporting, this is $\bar{x}\pm 0.98$. If $\bar{x}=1.2$, the realized confidence interval is approximately $[0.22,2.18]$. Exact 95% calibration refers to the unrounded standard normal quantile; using $1.96$ literally gives coverage probability $0.9500042097\dots$, almost but not exactly 0.95.

The confidence interval's width scales as $n^{-1/2}$: increasing sample size from 16 to 64 halves its half-width to $0.4899909961\dots$. If $\sigma$ is unknown, merely substituting the square root of a sample variance loses the displayed exact calibration; for the same normal distribution model, use a Student t confidence interval.

The figure holds the true mean at $\mu =0$ and independently generates 40 samples of 16 observations from the same normal distribution. NumPy's PCG64 generator with seed 42 gives 39 covering confidence intervals and one miss, at sample 32: observed coverage is $39/40=97.5\%$. Every bar uses the unrounded standard normal quantile. This finite collection illustrates the moving endpoints; it does not establish the analytic coverage probability or require exactly two misses.

The beauty of the construction is that a familiar standard normal distribution calibrates every possible unknown mean at once. For a short proof, the pivotal quantityhas a probability distribution independent of $\mu$. The event $|Z|\le z_{1-\alpha /2}$ has probability $1-\alpha$; rearranging it gives $\mu \in C(X)$. The right panel shows the equivalent event $|\bar{X}|\le 0.9799819923\dots$ when $\mu =0$.

For a one-sided normal-mean test of size $\alpha$ with known standard deviation $\sigma$, a positive alternative ${\delta }^{\ast }$ and target statistical power $1-\beta >\alpha$, the required sample size is the ceiling of ${\sigma }^2(z_{1-\alpha }+z_{1-\beta }{)}^2/({\delta }^{\ast }{)}^2$. This follows by shifting the standard normal quantile in the alternative rejection probability.

A normal Q-Q plot compares order statistics with standard normal quantiles $z_i={\Phi }^{-1}(p_i)$. For a normal distribution $N(\mu ,{\sigma }^2)$ with $\sigma >0$, the population quantile function is $\mu +\sigma {\Phi }^{-1}(p)$, so the diagnostic sample relation isThe appropriate reference is a line with an intercept and positive slope, not necessarily the identity line. A line can be fitted through two central sample quantiles; R's qqline uses the quantile levels $p=0.25$ and $p=0.75$ by default. Neither that line nor its fitting rule is part of the definition of a normal Q-Q plot. ^[ref]

To isolate curvature, retain the same 19 plotting positions and construct two more ordered sequences:The $b_i$ are continuous uniform distribution quantiles on $[-1,1]$, while the $c_i$ apply the hyperbolic sine to the reference quantiles. Both examples are symmetric about zero. At ranks $1,10,19$, $b_i$ is approximately $-0.947368,0,0.947368$ and $c_i$ is approximately $-1.812618,0,1.812618$. Again, these are constructed values rather than random draws or real data.

Read these shapes relative to their central slope. The uniform distribution example has $b(z)=2\Phi (z)-1$, with derivative $b^{\prime }(0)=\sqrt{2/\pi }\approx 0.797885$; at the upper endpoint the tangent predicts $1.546246$, while $b_{19}=0.947368$. Its lower end lies above the tangent and its upper end below it: the outer quantiles are compressed. The hyperbolic sine example has $c^{\prime }(0)=0.7$; its tangent predicts $1.356552$ at the upper endpoint, while $c_{19}=1.812618$. Its lower end lies below the tangent and its upper end above it: the outer quantiles are stretched. Unequal departures at the two ends can suggest skewness. The direction of these descriptions assumes that reference quantiles are horizontal and observed values vertical. The graphs illustrate finite-range shape differences; a sample pattern cannot establish an asymptotic tail probability law or membership in a heavy-tailed distribution class.

A nearly straight normal Q-Q plot is a diagnostic, not a proof of a normal distribution or a formal hypothesis test. Random statistical samples fluctuate, particularly at extreme order statistics, and apparent curvature depends on sample size, plotting positions and the reference line. A calibrated hypothesis test or uncertainty band requires its own assumptions and must account for parameters estimated from the same observations. Sorting also discards the original observation order: rearranging a time series leaves its Q-Q plot unchanged, so the plot cannot establish independence.

For a normal linear model, R's diagnostic normal Q-Q plot uses standardized regression residuals, also called internally studentized residuals. ^[ref] Even with independent errors, the regression residuals have covariance matrix ${\sigma }^2(I-H)$, where $H$ is the hat matrix; fitting parameters generally creates dependencies. Dividing by a shared estimated standard deviation creates further dependence, and the distribution of an internally studentized Gaussian residual is not exactly the standard normal distribution. The normal Q-Q plot therefore helps assess error shape alongside other diagnostics rather than certifying independent normal random variables.

For equal allocation, let $n$ be the required sample size per arm, $p_C=0.0045$, $p_T=0.003$ and $\Delta =0.0015$. In the standard sample size for comparing two proportions calculation, a two-sided test at 5% with 80% statistical power requiresUsing $z_{0.975}=1.95996$ and $z_{0.8}=0.84162$ gives $n\simeq 26063.64$, which is rounded upward for actual recruitment. ThusSince the risks are small, dropping the $(1-p)$ corrections and using $1.96+0.84$ gives $N_{\mathrm{total}}\simeq 2(2.8{)}^2(0.0075)/(0.0015{)}^2=52266.7$, conventionally about 52.3 thousand. These are slightly different approximations to the same planning requirement. Add an allowance if outcome ascertainment is incomplete or the allocation is clustered. A prespecified one-sided test would need a different, smaller sample size; the calculation here uses the conventional two-sided interpretation consistently with part (b).

Now $p_A=2/3$, $p_B=1/2$, so $\bar{p}=7/12$, $\Delta =1/6$, $v_0=35/72$ and $v_1=17/36$. The same sample size calculation givesAt $\alpha =0.05$, the illustrative two-sided total is $2\times 137=274$; the one-sided total is $2\times 108=216$. The smaller difference needs roughly four times the size: its squared magnitude is a quarter of the previous one, while the variance factors change comparatively little.

scan reads the numerical prices, including the three missing-value markers, into a vector. The data must be read row by row for the subsequent factor construction to match them. The sample size available for analysis is therefore $37$, rather than $40$. The numerical summary describes the observed prices: the sample median is £43.54 and the sample mean is £46.94. The upper tail, particularly the boots, raises the sample mean above the sample median. These summaries mix very different products and are not adjusted country comparisons.

The two character scans create labels. gl(8,5,length=40,labels=item) makes a regression factor with each of eight item labels repeated five times. gl(5,1,length=40,labels=country) cycles through the five country labels once for each item. For a regression factor and numerical response, the two plot commands produce grouped box plots. The country box plots suggest lower US prices, while the item box plots show particularly expensive boots and inexpensive cardigans. They are descriptive displays of this small basket, with different item compositions where prices are missing; they cannot separate country effects from item effects.

The first lm fits an additive two-way analysis of variance:The constraints are R's usual treatment coding. The normal linear model used for the printed tests assumes independent errors with common variance and approximately a normal distribution, and no item-by-country interaction in the mean. Missing prices are omitted, leaving $37$ responses and $1+4+7=12$ independent mean coefficients. Thus the residual degrees of freedom are $37-12=25$. This deletion is defensible for the conditional price model when missingness does not depend on the unobserved error after conditioning on the two factors; the output itself does not establish such a missing-data mechanism.

The anova tables report sequential sums of squares, rather than order-independent adjusted effects. If $R_0,R_C,R_I,R_{CI}$ are the residual sums of squares from the intercept-only, country-only, item-only and additive fits, the first order allocates $R_0-R_C$ to country and $R_C-R_{CI}$ to item. The second order allocates $R_0-R_I$ to item and $R_I-R_{CI}$ to country. Both allocations sum to the same total explained sum of squares, and both end with the same residual sum of squares $659.4$. The three missing combinations destroy balanced factorial orthogonality, so the allocation changes with order. To test country after allowing for items, use the country row in the second table; to test item after allowing for countries, use the item row in the first table. The first-entered rows do not supply those adjusted tests.

Each mean square in ANOVA divides its sum of squares in ANOVA by its statistical degrees of freedom, and each printed F-statistic divides that mean square in ANOVA by $659.4/25=26.38$. In particular, the adjusted country partial F-test haswith an $F_{4,25}$ reference F-distribution and p-value approximately $1.86\times 10^{-6}$. The adjusted item partial F-test gives $F=91.58$ on $7,25$ statistical degrees of freedom. Both factors contribute strongly under the additive normal linear model. The larger item sum of squares in ANOVA measures the much greater differences between these products, but is not evidence that every pair of items differs significantly.

The intercept $50.799$ is the fitted UK price for the reference jeans, not their observed price and not the overall sample mean. Each country regression coefficient is its adjusted price difference from the UK, assumed common across items: Sweden is £5.224 lower, France £10.708 lower, Germany £7.676 lower and the US £21.328 lower. Each item regression coefficient is its adjusted difference from the reference jeans, assumed common across countries. Thus the fitted UK boots price is $50.799+52.854=103.653$, and the fitted US boots price is $103.653-21.328=82.325$.

The coefficient standard errors come from $s^2(X^{T}X{)}^{-1}$, where $s^2=659.4/25$ and $X$ is the design matrix. Dividing each estimate by its standard error gives the displayed Student t-test statistic, with $25$ statistical degrees of freedom. The stars summarize the corresponding two-sided p-values. The individual tests compare each indicated country or item with its reference level, conditional on all other additive terms; they are neither tests of every pairwise comparison nor a simultaneous confidence interval procedure. In particular, Sweden's p-value $0.0626$ provides weaker evidence than the other country contrasts, and does not establish equality with the UK. The intercept test has little substantive relevance, and interpreting many individual stars also raises multiple hypothesis testing issues.

The residual standard error $s=5.136$ measures unexplained prices in pounds. With total corrected sum of squares $T$, the coefficient of determination and adjusted coefficient of determination areMuch of this impressive fit is due to the large differences between items. The overall F-test compares the additive fit with a constant mean: $F=((T-659.4)/11)/(659.4/25)=62.12$ on $11,25$ statistical degrees of freedom. It tests that all eleven non-intercept coefficients vanish, not that the additive model is adequate. Regression diagnostics should examine regression residuals, changing price variability across items and possible interaction. Percentage differences may be more plausible than common pound differences, motivating a natural logarithm of price. No random sample of countries or products, and no independent replication of a price within a cell, is supplied; broad claims about all designer goods or the causes of country differences require more than these conditional tests.

Finally, Item*Country expands to both main effects and their full interaction. A cell-means parameterization has one separate fitted mean for every observed cell. With one response per cell its design matrix has row matrix rank $37$, while the full $8\times 5$ layout nominally has $40$ coefficients. Three coefficient combinations are aliased because their cells are missing. This is saturation of an unreplicated two-factor regression: every observed price is fitted exactly and the residual degrees of freedom are zero. The final fit has zero residuals, $R^2=1$ and no residual-based estimate of the error variance or ordinary interaction significance test. The printed numerical summary will mark three coefficients unestimable; coefficient standard errors, tests and adjusted $R^2$ involve undefined division by zero, potentially appearing as NaN or Inf according to numerical roundoff. Perfect interpolation here is a consequence of a saturated statistical model, not evidence for a superior predictive model.

A court-based randomized controlled trial would assign eligible offenders between the two sentencing policies using randomization, ideally stratified by court and important prespecified risk factors. With allocation concealment, the next assignment cannot be selected to suit an offender's prognosis. Baseline characteristics, both recorded and unrecorded, are balanced in distribution rather than confounded with judges' sentencing preferences. This makes a comparison of outcomes interpretable as a causal effect of assignment, subject to complete and comparable follow-up, whereas separate historical cohorts can differ in severity, social circumstances or calendar period.

The commission should compare both specified outcomes over the same 360-day period from sentencing, using an intention-to-treat analysis. An offender initially assigned community service remains in that group even if the order is later revoked and prison follows. Revocation is part of the policy's consequences; restricting the comparison to completers would select on a post-assignment outcome and lose the protection of randomization.

Within-court individual randomization also provides an objective allocation procedure and valid uncertainty estimates. It does not guarantee identical characteristics in a particular finite sample or eliminate loss to follow-up, unequal outcome ascertainment or limited generalizability. Eligibility, genuine uncertainty about the two policies, and a feasible authorized allocation mechanism would need agreement before the trial. If whole courts rather than individuals were assigned, that would be a cluster-randomized trial and its correlation structure would require a different sample size calculation.

Assume equal allocation of $n$ independent offenders to each arm and a two-sided statistical test of significance level $\alpha$. Write $p_0=0.60$ for P30 and $p_1=0.56$ for initial community-service assignment, with $\Delta =p_0-p_1=0.04$ and $\bar{p}=(p_0+p_1)/2=0.58$. The two event counts have independent binomial distributions. Under the stated alternative, the difference of sample proportions is approximately a normal distribution with mean $\Delta$ and varianceThe pooled two-proportion test has approximate upper rejection threshold $z_{1-\alpha /2}\sqrt{2\bar{p}(1-\bar{p})/n}$ at the planning rates, where $z_q$ is the standard normal quantile. To obtain statistical power $0.80$ against a positive difference, place the alternative mean $z_{0.80}$ alternative-standard-errors above that threshold:Solving yields the usual sample size for comparing two proportions:per arm, rounded upwards, with total sample size $2n$. This targets the dominant rejection tail under the planning alternative; an exact power calculation can include the very small opposite-tail probability or use the binomial distributions directly.

For illustration, at $\alpha =0.05$, $z_{0.975}\simeq 1.960$ and $z_{0.80}\simeq 0.842$, giving $n\simeq 2388.8$. ThusFor a prespecified one-sided test, replace $z_{1-\alpha /2}$ by $z_{1-\alpha }$. The calculation assumes complete endpoint ascertainment, independent individual allocation and that the two initial-policy rates are appropriate; losses or clustered allocation would require corresponding design adjustments.

Mortality requires its own sample size calculation, rather than using the much commoner reconviction endpoint. Under the same specified exposure assumptions, the P30 schedule has 30 prison days, 30 high-risk post-release days, and 300 ordinary days. Its integrated hazard isHence its 360-day death probability is $p_P=1-e^{-0.0145}\simeq 0.0143954$. For initial community-service assignment the preceding mixture gives $p_C\simeq 0.0127180$. The absolute risk difference is therefore only about $0.0016774$, or 16.8 fewer deaths per 10,000 initial assignments. The person-time approximation gives 145 versus 128 deaths per 10,000, a difference of 17.

As an illustrative design target, take two-sided significance level $0.05$, statistical power $0.80$ and equal individual allocation. Apply the sample size for comparing two proportions formula with these mortality probabilities:It gives about 74,600 participants per arm, or 149,200 altogether; using the simpler exposure approximations gives about 73,130 per arm. At 30,000 sentences per year, even if every offender were eligible and enrolled, this requires roughly five years of recruitment and another year to complete the final cohort's follow-up.

For scale, one year's equally allocated cohort has 15,000 per arm, with roughly 216 expected deaths after P30 and 191 after community-service assignment. The standard error of the risk difference is about $0.001335$, so the expected standardized separation is only about $1.26$, below the usual two-sided 5% critical value. A one-year trial would consequently have poor statistical power for this mortality difference.

Thus mortality discrimination is much more demanding than reconviction discrimination. National recruitment and reliable linkage to death records could make a multiyear study feasible, but restricted eligibility, incomplete follow-up, or lower recruitment would lengthen it. A trial sized only for the 4-percentage-point reconviction difference would not provide a precise mortality comparison. The numerical conclusion depends on the supplied daily hazards, custody schedules, and the chosen power and significance targets; the prompt does not specify a mortality design target of its own.