Suppose are independent and identically distributed with normal distribution , where the mean is unknown and the standard deviation is known. The sample mean has normal distribution and standard error . With the standard normal quantile at , the displayed confidence interval has exact coverage probability for every and every positive integer . No central limit theorem approximation is needed.
Take , and nominal confidence 95%. The standard error is , and . ThereforeRounded for reporting, this is . If , the realized confidence interval is approximately . Exact 95% calibration refers to the unrounded standard normal quantile; using literally gives coverage probability , almost but not exactly 0.95.
The confidence interval's width scales as : increasing sample size from 16 to 64 halves its half-width to . If is unknown, merely substituting the square root of a sample variance loses the displayed exact calibration; for the same normal distribution model, use a Student t confidence interval.
The figure holds the true mean at and independently generates 40 samples of 16 observations from the same normal distribution. NumPy's PCG64 generator with seed 42 gives 39 covering confidence intervals and one miss, at sample 32: observed coverage is . Every bar uses the unrounded standard normal quantile. This finite collection illustrates the moving endpoints; it does not establish the analytic coverage probability or require exactly two misses.
The beauty of the construction is that a familiar standard normal distribution calibrates every possible unknown mean at once. For a short proof, the pivotal quantityhas a probability distribution independent of . The event has probability ; rearranging it gives . The right panel shows the equivalent event when .
For the same normal distribution model, a two-sided test compares the null hypothesis with the alternative hypothesis . Under the null hypothesis, the standardized test statistic has standard normal distribution. Write for the standard normal distribution function. With , , and , the observed test statistic is . Its p-value isThus the p-value is a tail probability computed under the null hypothesis, including departures in both directions. It is not , the posterior probability of the null hypothesis, nor the probability that the observations arose "by chance". The normal distribution test statistic and its calibration appear in NIST's teaching reference for testing a process mean; the tail interpretation is stated in Penn State's hypothesis-testing notes.
Choose the significance level before observing the sample. The rejection region is , whose probability under is exactly 0.05. Here , so reject . This agrees with zero lying outside . More generally, a proposed belongs to the unrounded 95% confidence interval precisely when its two-sided test does not reject under this strict cutoff. The equality boundary has probability zero in this continuous statistical model.
The significance level and statistical power describe different repeated-experiment probabilities. At a specified alternative , put and . ThenThe first is the Type I error probability; the second is the Type II error probability at that alternative. For a planning alternative , keeping , and gives , and statistical power . At , and statistical power rises to . These are model calculations for specified alternatives, not posterior probabilities inferred from a nonsignificant result. They explain why failure to reject cannot establish equality and why choosing sample size before an experiment is useful.
Historically, Ronald Fisher's 1925 book discusses the normal cutoff and tail probability 0.05 as a convenient convention on page 47, rather than a mathematically compulsory threshold: [ref]
https://books.google.com/books/about/Statistical_Methods_for_Research_Workers.html?id=I0NBAAAAIAAJ. Fisher, Statistical Methods for Research Workers, first edition
. Articles by others on the same topic
There are currently no matching articles.

