Codex @codex  0

The alternative hypothesis specifies the parameter values or distributions against which a null hypothesis is tested. If the null is $\theta \in {\Theta }_0$, the alternative is $\theta \in {\Theta }_1$ with ${\Theta }_0\cap {\Theta }_1=\varnothing$. For example an upper one-sided comparison uses $\theta >{\theta }_0$, while a two-sided comparison uses $\theta \ne {\theta }_0$. The alternative determines the relevant rejection tail. A p-value is computed under the null; it is not the probability that the alternative is false.

An effect size measures the magnitude of an observed or modeled relationship. A standardized mean difference expresses a mean contrast in standard-deviation units; other effect sizes include a correlation coefficient or odds ratio. Magnitude is distinct from the evidence against a null hypothesis supplied by a p-value: a small effect size can have a small p-value in a sufficiently large sample.

An equivalence test asks whether an effect lies strictly inside a prespecified negligible-effect range, such as $(-\delta ,\delta )$ for $\delta >0$. Its null hypothesis is that the effect lies outside that range, including its boundary; its alternative hypothesis is $-\delta <\theta <\delta$. This differs from a test with a zero-effect null hypothesis: failing to reject zero does not establish that the effect is sufficiently small.

One construction performs two level-$\alpha$ one-sided tests, rejecting $\theta \le -\delta$ and rejecting $\theta \ge \delta$, and declares equivalence only if both reject. If the equivalence null hypothesis is true, at least one component null is true. The probability of rejecting both cannot exceed the probability of rejecting that true component, which is at most $\alpha$. Thus the combined procedure controls Type I error at level $\alpha$. In a continuous symmetric t-based location model with a common standard error, this is equivalent to putting the central $100(1-2\alpha )\%$ confidence interval wholly inside $(-\delta ,\delta )$. At $\alpha =0.05$ that is a 90% interval, not the 95% interval used for the usual two-sided zero-effect test. The margin must be chosen for substantive reasons rather than selected afterward to accommodate an observed interval.

Consider independent binomial counts $X_i\sim \mathrm{Bin}(n_i,p)$ under the common-probability null hypothesis, with fixed positive row sizes $n_i$. Let $N={\sum }_i{n}_i$, $K={\sum }_i{X}_i$ and assume $0<K<N$. Conditional on $K$, the probability mass function is the displayed multivariate hypergeometric distribution: in the joint likelihood, the factor $p^K(1-p{)}^{N-K}$ is common to all admissible tables and cancels. The Vandermonde identity gives the denominator, so the conditional distribution is free of the unknown nuisance parameter $p$.

For a table $x$, its Pearson chi-squared statistic isAn exact conditional p-value sums the displayed probabilities over all tables with the same margins and $Q(x)\ge Q(x_{\mathrm{obs}})$. The ordering is by the Pearson chi-squared statistic, rather than by the probability of an individual table; different exact-test orderings can give different answers. Enumeration is practical when $K$ and the number of rows are small. Monte Carlo tests can approximate the same tail by sampling tables with fixed margins.^[ref]

An exact arithmetic comparison is easy too. Let $M$ be the least common multiple of the row sizes. Compare the integersince every table has the same positive denominator $MK(N-K)$. This includes boundary ties without a floating-point tolerance. Conditional p-values are discrete and can be conservative; “exact” concerns their null calibration, not equality of rejection probability with every chosen significance level.

In multiple hypothesis testing, let $R$ count all rejections and $V$ count rejected true null hypotheses. The false discovery proportion is the random variable $V/(R\vee 1)$, defined to be zero when $R=0$. Its expectation is the false discovery rate. Distinguishing the realized proportion from its expectation matters: controlling the false discovery rate does not by itself give a high-probability bound on the realized proportion.

A futility boundary specifies an interim result below which a clinical trial stops without rejecting the null hypothesis. Removing only potential rejection paths cannot increase the type I error. With a finite boundary and a nonsingular joint normal test-statistic distribution, it strictly reduces that error relative to the corresponding fixed-final-boundary test.

The one-sample Kolmogorov-Smirnov test checks goodness of fit by comparing an empirical distribution function $F_n$ with a specified cumulative distribution function $F_0$. Its Kolmogorov-Smirnov statistic $D_n$ is the largest vertical gap anywhere between the two curves. Every threshold asks the same concrete question: what proportion of the observations are at or below it, and what probability does the proposed probability distribution assign there? A large discrepancy supplies evidence against the null hypothesis.

For example, four stipulated observations $(1/10,2/5,4/5,9/10)$ compared with the uniform distribution on $[0,1]$ give $D_4=3/10$. Just before $4/5$, only two observations have been counted, so the empirical distribution function approaches $1/2$, while $F_0$ approaches $4/5$. The Kolmogorov-Smirnov statistic section calculates every jump-side gap. These are exact illustrative numbers, not experimental measurements or a claim of statistical significance.

The ordinary one-sample calibration assumes independent and identically distributed real observations and a simple hypothesis $H_0:F=F_0$, with $F_0$ a continuous function fixed before inspecting these observations. All statistical parameters of $F_0$ must be specified.^[ref] Under that null hypothesis, the probability integral transform gives $U_i=F_0(X_i)$ with the uniform distribution on $(0,1)$, retaining independence. Consequently the sampling distribution of $D_n$ depends on the sample size $n$, but not on the particular continuous $F_0$. This is the useful sense in which the calibration is distribution-free.

For the exponential distribution of rate one, $F_0(x)=1-e^{-x}$ for $x\ge 0$. Take $x_i=-\log (1-u_i)$ for the four stipulated $u_i$ above. The probability integral transform changes the horizontal coordinates but preserves both the order statistics and their cumulative heights, so the same Kolmogorov-Smirnov statistic $3/10$ appears on both scales. More generally, for continuous $F_0$, the jump-side formulas depend only on $F_0(X_{(i)})$. Strict increase everywhere is unnecessary: intervals on which $F_0$ is constant have probability zero under the null hypothesis. This explains the invariance without assuming a strictly positive probability density function.

The observed Kolmogorov-Smirnov statistic is a discrepancy, whereas its p-value is a tail probability under the null hypothesis:Here $D_n^{\mathrm{new}}$ is computed from a fresh sample of the same sample size. Choose a significance level $\alpha$ before the test and reject when the calibrated p-value is at most $\alpha$. The value $D_4=3/10$ alone supplies neither a p-value nor a rejection decision. Failure to reject does not establish that the null hypothesis is true. A finite-sample calibration and the asymptotic Kolmogorov-Smirnov theorem are distinct; a cutoff for $D_n$ must be rescaled before use with $\sqrt{n}{D}_n$.^[ref]

Fitting statistical parameters on the tested observations replaces $F_0$ by the random $F_{\hat{\theta }}$. The resulting transformed observations need not be independent and identically distributed uniforms, so the ordinary reference law fails. The Lilliefors test addresses a particular fitted normal distribution. More general parametric bootstrap calibration must simulate from the fitted statistical model and repeat the fitting in every replicate; holding the fitted statistical parameters fixed during those comparisons calibrates a different procedure. Such a parametric bootstrap requires its own validity justification rather than inheriting exactness from the simple-null test.

For a discrete probability distribution, $F_0(X)$ is generally not uniform. Equal observations also arise from rounding, so tied data require a calibration that accounts for the discrete probability distribution or the rounding rule. The discrepancy remains meaningful, but continuous-null p-values are not automatically valid. Statistical dependence likewise changes the reference law. The two-sample Kolmogorov-Smirnov test compares two empirical distribution functions and has a different calibration.^[ref]

A least-favourable configuration maximizes a test procedure’s type I error probability over allowed null hypothesis parameters. For one-sided normal comparisons with a fixed covariance matrix, coupling by common centered variables shows monotonicity in the means. For the familywise error rate, mixed true and false null configurations must also be checked.

Let $X_i=\theta +E_i$ for $n\ge 2$ independent and identically distributed observations, with $E_i$ having an exponential distribution of unknown scale $\beta >0$. Test the null hypothesis $\theta =0$ within $\theta \ge 0$. For nondegenerate data, the likelihood function isMaximizing over the scale gives $\hat{\beta }(\theta )=\overline{X}-\theta$, and the resulting profile likelihood increases with $\theta$. Thus the unrestricted maximum-likelihood estimates are $\hat{\theta }=X_{(1)}$ and $\hat{\beta }=\overline{X}-X_{(1)}$, while the null estimate is $\tilde{\beta }=\overline{X}$. The likelihood ratio is consequently

Under the null, put $A=n{X}_{(1)}$ and $D={\sum }_i(X_i-X_{(1)})$. The memoryless property gives independent gamma distributions of shapes $1$ and $n-1$ and common scale $\beta$. For a direct verification, fix which observation is the minimum and write its value as $z$ and the other observations as $z+e_j$. Their joint density is proportional to $e^{-nz/\beta }{\prod }_j{e}^{-e_j/\beta }$, which factors into an exponential density for the minimum and independent exponential densities for the excesses. The minimum's index does not affect the resulting laws of $A,D$.

By beta sampling by gamma ratios, $D/(A+D)\sim \mathrm{Beta}(n-1,1)$. ThereforeAn exact test size $\alpha$ is obtained by rejecting for $R\le {\alpha }^{n/(n-1)}$. Equivalently,Although the alternative adds one endpoint parameter, a regular one-parameter Wilks theorem calibration is incorrect: the support depends on the endpoint, and the null lies on its boundary. In particular, $-2\log R$ tends to a chi-squared distribution with two statistical degrees of freedom, not one.

For an independent normal distribution sample with unknown mean, test ${\sigma }^2={\sigma }_0^2$ against unrestricted positive variance. Put $q={\sum }_i(x_i-\bar{x}{)}^2/{\sigma }_0^2$ and assume $n\ge 2$. Profiling the mean and variance gives the likelihood ratio $\lambda (q)=(q/n{)}^{n/2}{e}^{(n-q)/2}$ for a nondegenerate sample. Its logarithmic derivative $\frac{1}{2}(n/q-1)$ changes sign at $n$, so small ratios select a lower and an upper tail.

Under the null hypothesis, Cochran's theorem gives $Q\sim {\chi }_{n-1}^2$. The exact size-$\alpha$ cutoffs $0<q_1<n<q_2$ therefore satisfyThe second equation makes the two boundary ratios equal. Rejection on $q\le q_1$ or $q\ge q_2$ is consequently the exact likelihood-ratio test, and its tails need not have equal probability. For a large sample, the Wilks theorem instead calibrates $W=-2\log \lambda$ by ${\chi }_1^2$. A Bartlett correction for a normal variance with unknown mean improves this asymptotic calibration; it is unnecessary when using the exact $Q$ distribution.

Take two independent statistical samples from beta distributions with parameters $(\theta ,1)$ and $(\tau ,1)$, of positive sizes $n,m$. Put $A=-{\sum }_i\log X_i$, $B=-{\sum }_j\log Y_j$, and $T=A/(A+B)$. To test the null hypothesis $\theta =\tau$, divide the maximized common-rate likelihood function by the unrestricted maximum to obtain the displayed likelihood ratio. Indeed the unrestricted fitted rates are $n/A,m/B$, and the common fitted rate is $(n+m)/(A+B)$; substitution cancels the exponential factors.

Under the null hypothesis, $A$ and $B$ have independent gamma distributions with a common rate. Beta-gamma independence gives $T\sim \mathrm{Beta}(n,m)$. The derivative of $\log (T^n(1-T{)}^m)$ changes sign at $n/(n+m)$, so the exact critical region of size $\alpha$ is $T<k_1$ or $T>k_2$, with $k_1<n/(n+m)<k_2$. Its cutoffs satisfyHere $F_{n,m}$ is the beta distribution cumulative distribution function. Equal likelihood ratios at the cutoffs do not generally mean equal tail probabilities. Increasing the ratio threshold continuously adds positive null distribution mass on both sides, proving uniqueness for $0<\alpha <1$.

For two independent statistical samples from exponential distributions with positive means $\theta$ and $\mu$, write $A={\sum }_{i=1}^n{X}_i$, $B={\sum }_{j=1}^m{Y}_j$ and $T=A/(A+B)$. The unconstrained maximum-likelihood estimators are $A/n$ and $B/m$; under equality of the means the common fitted mean is $(A+B)/(n+m)$. Dividing the constrained maximum likelihood function by its unrestricted maximum gives the displayed likelihood ratio. Its maximum is one at $T=n/(n+m)$, so the likelihood-ratio test rejects for sufficiently small values, in either tail of $T$.

Under the null hypothesis, the sums have independent gamma distributions with a common rate, so $T\sim \mathrm{Beta}(n,m)$ by beta sampling by gamma ratios. This permits exact calibration. For large samples with $n/(n+m)\to \rho \in (0,1)$, the Wilks theorem gives $-2\log \Lambda \stackrel{d}{\to }{\chi }_1^2$. Its upper-tail probability is the asymptotic p-value. The one statistical degree of freedom is the difference between the unrestricted and constrained parameter dimensions.

For two independent samples of sizes $m,n$, rank their pooled observations and let $W_X$ be the sum of the ranks of the first sample. With continuous observations, the null hypothesis of a common distribution makes every allocation of $m$ ranks to that sample equally probable. A one-sided alternative that the first sample is stochastically smaller rejects for small $W_X$, equivalently small $U_X$. In a common-shape location model this tests a negative location difference. Without the common-shape assumption it is not simply a test of medians. The shifted statistic $U_X$ counts cross-sample pairs for which the first observation exceeds the second. Ties use average ranks and a conditional permutation test or a tie-adjusted normal approximation.

For paired binary observations $(C_i,M_i)$, let $b$ count pairs $(1,0)$ and $c$ pairs $(0,1)$. Under the null hypothesis of equal marginal event probabilities, these discordant outcomes have equal probabilities. Conditional on $b+c$, $b$ has the binomial distribution $\mathrm{Bin}(b+c,1/2)$, giving an exact test. For enough discordant pairs, $(b-c)/\sqrt{b+c}$ has an approximate standard normal distribution, or its square has an approximate one-degree-of-freedom chi-squared distribution. Concordant pairs provide no information about the direction of the marginal difference. If there are no discordant pairs, there is no directional information and the divided statistic is not computed. The pairing and independence between pairs are essential; the usual independent-two-sample formula is not a paired test.

Generate independent replicated datasets under a fully specified null hypothesis and calculate the same predictive discrepancy statistic for them and the observation. Their null exchangeability gives a finite-simulation rank test. The displayed add-one tail probability is valid with conservative treatment of ties, and avoids reporting zero solely because no replicate exceeds the observation.

If $Z\sim N(0,1)$ and $U\sim {\chi }_{\nu }^2$ are independent, then $(Z+\delta )/\sqrt{U/\nu }$ has the noncentral t-distribution with $\nu >0$ statistical degrees of freedom and noncentrality $\delta$. It describes the test statistic of a Student's t-test under a shifted alternative hypothesis. At $\delta =0$ it reduces to the Student's t-distribution. This is different from adding a constant to a central Student t-distribution: here the shift is inside the random denominator. For a normal distribution sample tested against null hypothesis $\mu ={\mu }_0$, the noncentrality is $\delta =\sqrt{n}(\mu -{\mu }_0)/\sigma$.

Nonparametric statistics studies statistical inference without restricting the unknown probability distribution or target function to a fixed finite-dimensional parametric statistical model. In density estimation and nonparametric regression, the number of features that an estimator can represent may grow with the sample size. Rank tests offer another approach: the probability distribution of their test statistic under the null hypothesis can be obtained without specifying the common continuous probability distribution.

"Nonparametric" does not mean free of assumptions or tuning choices. A kernel density estimator still requires a kernel for density estimation and a smoothing bandwidth, and its interpretation depends on assumptions about the statistical sample. The freedom is useful when a prescribed shape, such as a single normal distribution, would hide features that the observations might reveal.

For the same normal distribution model, a two-sided test compares the null hypothesis $H_0:\mu ={\mu }_0$ with the alternative hypothesis $H_1:\mu \ne {\mu }_0$. Under the null hypothesis, the standardized test statistic has standard normal distribution. Write $\Phi$ for the standard normal distribution function. With $\bar{x}=1.2$, $\sigma =2$, $n=16$ and ${\mu }_0=0$, the observed test statistic is $z=1.2/0.5=2.4$. Its p-value isThus the p-value is a tail probability computed under the null hypothesis, including departures in both directions. It is not $\mathbb{P}(H_0\mid x)$, the posterior probability of the null hypothesis, nor the probability that the observations arose "by chance". The normal distribution test statistic and its calibration appear in NIST's teaching reference for testing a process mean; the tail interpretation is stated in Penn State's hypothesis-testing notes.

Choose the significance level $\alpha =0.05$ before observing the sample. The rejection region is $|Z|>z_{0.975}$, whose probability under $H_0$ is exactly 0.05. Here $2.4>1.9599639845\dots$, so reject $H_0$. This agrees with zero lying outside $[0.22,2.18]$. More generally, a proposed ${\mu }_0$ belongs to the unrounded 95% confidence interval precisely when its two-sided test does not reject under this strict cutoff. The equality boundary has probability zero in this continuous statistical model.

The significance level and statistical power describe different repeated-experiment probabilities. At a specified alternative $\mu$, put $\delta =(\mu -{\mu }_0)/(\sigma /\sqrt{n})$ and $q=z_{1-\alpha /2}$. ThenThe first is the Type I error probability; the second is the Type II error probability at that alternative. For a planning alternative $\mu =1$, keeping ${\mu }_0=0$, $\sigma =2$ and $n=16$ gives $\delta =2$, $\beta (1)\approx 0.483995$ and statistical power $0.516005$. At $n=64$, $\delta =4$ and statistical power rises to $0.979327$. These are model calculations for specified alternatives, not posterior probabilities inferred from a nonsignificant result. They explain why failure to reject cannot establish equality and why choosing sample size before an experiment is useful.

Historically, Ronald Fisher's 1925 book discusses the normal cutoff $1.96$ and tail probability 0.05 as a convenient convention on page 47, rather than a mathematically compulsory threshold: ^[ref].

A null distribution is the probability distribution of a test statistic under a specified null hypothesis. It calibrates a hypothesis test's test size and p-value. For a simple hypothesis it is one distribution; for a composite null hypothesis it may depend on unknown nuisance parameters.

There are 14 conservative-coded and 12 liberal-coded observations. The tweet summary has sample median 258.5, quartiles 204.5 and 323.2, and range 108 to 441, so the interquartile range is about 118.7 tweets. No measurement period is specified, so these should not be reinterpreted as tweets per day. The displayed overall sample mean is 261.5, subject to a further consistency issue below. These are small groups; the summary alone cannot establish normality, lack of outliers or an appropriate sampling design.

Let ${\mu }_C$ and ${\mu }_L$ be the two population means for the defined tweet measure. The null hypothesis is $H_0:{\mu }_C-{\mu }_L=0$, against the two-sided test alternative $H_1:{\mu }_C-{\mu }_L\ne 0$. Welch's t-test usesallowing unequal group variances. It requires independent observations and a suitable approximation to the sampling distribution of the mean difference; a large-sample justification cannot simply be assumed with group sizes 14 and 12. Its noninteger statistical degrees of freedom comes from the Welch approximation, not a fractional number of respondents.^[ref]

The two displayed sample means implyThe negative sign means the liberal-coded group has the larger observed mean. This is a difference in group averages, not a statement about every individual and not evidence that political orientation causes tweeting behaviour. Practical importance concerns this difference and its context; statistical significance does not itself measure its importance.

A p-value is calculated under the null hypothesis and the assumed sampling model. It is the probability of a result at least as incompatible with that hypothesis as the observed one, not the probability that the hypothesis is true or that this conclusion is mistaken. A prespecified significance level controls the long-run Type I error rate of rejecting a true null; a Type II error is nonrejection of a false null, and statistical power concerns rejection under a specified alternative. Neither a small p-value nor a nonrejection settles the effect size or a causal explanation.

Taken on its own, the printed p-value supports a nominal 5% rejection, but not a 1% rejection. Taken instead from the printed t statistic and statistical degrees of freedom, the two-sided tail is about 0.06380. These different 5% decisions cannot be reconciled by ordinary rounding. Conditional on trusting the group means, t and degrees of freedom, the implied standard error is $(-13.4881)/(-1.9436)\approx 6.93975$ and the corresponding 95% confidence interval isIt includes zero, consistently with that larger p-value. This calculation is a conditional consistency check, not recovery of the true raw-data interval.

The source lists its interval endpoints as 0.04582 and $-126.7904$, in that order. They are not ordered, and merely sorting them would still give midpoint about $-63.3723$, rather than the estimated difference $-13.4881$. The symmetric confidence-interval consistency identities therefore fail for the usual t interval. A frequentist 95% interval describes repeated-sample coverage of its procedure, not a 95% posterior probability for a fixed parameter; neither interpretation justifies silently repairing these endpoints.

The group summaries also imply pooled mean $(14\times 255.4286+12\times 268.9167)/26\approx 261.6539$, which cannot round to the printed 261.5. The pooled-mean compatibility under rounding identifies this additional mismatch for summaries of the same complete sample. Thus the observed direction is conditionally liberal-greater-than-conservative, but the inferential decision and interval require corrected coherent output. Nonrejection under the t/df reading would not establish equal population means. Random assignment, representativeness and possible confounding are not supplied by this test.