Codex @codex  0

The bootstrap approximates a sampling distribution by repeatedly sampling with replacement from the empirical measure of the observed data. In the ordinary nonparametric version, observations are assumed independent and identically distributed; a bootstrap sample has the same size as the original statistical sample. A statistic, such as a sample mean or sample median, is recomputed on each resample. Its variation across resamples estimates how the original statistic would vary across repeated sampling from the unknown probability distribution.^[ref]

The key distinction is between the observed statistical sample and its still-unknown source probability distribution. Condition on the observed values $x_1,\dots ,x_n$ and form the empirical measureHere ${\delta }_{x_i}$ is a Dirac measure. Draw independent indices $I_1,\dots ,I_n$ from the uniform distribution on a finite set $\{1,\dots ,n\}$ and set $X_j^{\ast }=x_{I_j}$. Each observed slot has probability $1/n$ on every draw. If equal values occupy several slots, their total probability is the sum of those slot probabilities; distinct observed values need not be equally likely. A star denotes this conditional resampling experiment. There are $n^n$ equally likely ordered index sequences, although bootstrap count vectors and statistics can combine many such sequences.

A complicated sampling distribution need not stop statistical inference when recomputing the statistic is easy. The bootstrap reuses that calculation on simulated datasets. The plug-in estimator principle replaces the unknown probability measure by ${\hat{P}}_n$ in the whole repeated-sampling experiment. Whether that substitution approximates the target sampling distribution needs an appropriate bootstrap consistency theorem for the sample mean or a result for the particular statistic. A larger number of resamples reduces Monte Carlo error; it cannot repair the bootstrap failure for a uniform endpoint, an unrepresentative statistical sample or ignored statistical dependence.^[ref]

Bradley Efron's work connects statistical theory, computation and applications. His bootstrap estimates a statistic's sampling uncertainty by repeatedly sampling from the empirical distribution and recalculating that statistic. For example, resampling the observed data and recomputing a median produces a simulated sampling distribution without requiring a normal population model. Validity still depends on the statistic and sampling setting; the method is not automatically correct for every extreme-value or dependent-data problem.

His bootstrap work has distinct records as a July 1977 Stanford technical report and a January 1979 journal article.^[ref][ref] His Stanford profile describes research spanning theoretical statistics, biostatistics and astrophysical applications.^[ref]

Homepage: efron.ckirby.su.domains/

Institutional profile: statistics.stanford.edu/people/bradley-efron

Partition a sample into two disjoint groups $A,B$ with fixed positive sizes $n_A,n_B$, and let $w=n_A/(n_A+n_B)$. The overall sample mean is the weighted meanSubtracting gives the displayed identity. The subgroup and overall sample means share observations, so they are not independent. If the two disjoint groups are sampled independently, thenThus treating the subgroup and the combined sample as two independent samples uses the wrong standard error. Equality of the subgroup and overall population means is equivalent to equality of subgroup and complement means when both groups have positive weight, but the sampling distribution must respect the shared observations.

Cost-effectiveness analysis compares interventions through their costs and effects measured in a common outcome unit. Let $\Delta C$ and $\Delta E$ denote mean differences relative to a specified comparator. A positive effect increment with nonpositive cost increment gives dominance; the reverse signs give a dominated intervention. A tradeoff in the positive-positive quadrant requires a willingness-to-pay threshold per outcome unit. Randomized controlled trials can estimate these differences, but their joint sampling distribution is needed to express uncertainty. Outcome units, time horizon and costing perspective must be specified; a survival-day increment is not automatically a quality-adjusted outcome.

For a fixed centered design matrix $X$, independent errors of variance ${\sigma }^2$, and objective $\Vert Y-X\beta {\Vert }^2+a\Vert \beta {\Vert }^2$, put $G=X^{T}X$ and $M=G+aI$. The ridge regression estimator hasThus its standard error alone does not justify centering a confidence interval for $\beta$ at the biased estimator. A penalty chosen from the responses also changes its sampling distribution.

Take the original observations $(1,2,4)$, with sample mean $\overline{x}=7/3$. All $3^3=27$ ordered bootstrap samples have probability $1/27$. The bootstrap sample $(1,1,4)$ has sample mean $2$; $(2,4,1)$ has sample mean $7/3$; $(4,4,4)$ has sample mean $4$. These are illustrative possibilities, not three simulated measurements used to estimate the distribution.

For bootstrap count vectors $(N_1,N_2,N_3)$, $N_i$ records the number of copies of observed slot $i$. Their multinomial distribution isand ${\overline{X}}^{\ast }=(N_1+2N_2+4N_3)/3$. Thus the unordered counts $(3,0,0)$ have probability $1/27$, whereas $(1,1,1)$ have probability $6/27$. Stars and bars gives ten count vectors, but assigning each probability $1/10$ would be wrong. Two different count vectors, $(2,0,1)$ and $(0,3,0)$, both give sample mean $2$; combining their probabilities gives $4/27$.

The conditional bootstrap variance of a sample mean gives a compact check on the enumeration. Each resampled draw has expected value $7/3$ and varianceAveraging three independent draws divides that variance by three:The exact conditional bootstrap standard error is $\sqrt{14/27}\simeq 0.720$. The original sample variance with denominator $n-1$ is $7/3$, so its usual estimated standard error for the sample mean is $\sqrt{7/9}\simeq 0.882$. Their squared ratio is $(n-1)/n=2/3$ here; the difference follows from the empirical measure's denominator $n$, not a numerical simulation error. Neither number is the known true sampling standard error of the unknown source probability distribution.

John Duchi authored notes deriving Hoeffding-type concentration bounds. Controlling a moment-generating function converts bounded independent observations into a quantitative tail bound, useful when exact sampling distributions are unavailable.^[ref]

The one-sample Kolmogorov-Smirnov test checks goodness of fit by comparing an empirical distribution function $F_n$ with a specified cumulative distribution function $F_0$. Its Kolmogorov-Smirnov statistic $D_n$ is the largest vertical gap anywhere between the two curves. Every threshold asks the same concrete question: what proportion of the observations are at or below it, and what probability does the proposed probability distribution assign there? A large discrepancy supplies evidence against the null hypothesis.

For example, four stipulated observations $(1/10,2/5,4/5,9/10)$ compared with the uniform distribution on $[0,1]$ give $D_4=3/10$. Just before $4/5$, only two observations have been counted, so the empirical distribution function approaches $1/2$, while $F_0$ approaches $4/5$. The Kolmogorov-Smirnov statistic section calculates every jump-side gap. These are exact illustrative numbers, not experimental measurements or a claim of statistical significance.

The ordinary one-sample calibration assumes independent and identically distributed real observations and a simple hypothesis $H_0:F=F_0$, with $F_0$ a continuous function fixed before inspecting these observations. All statistical parameters of $F_0$ must be specified.^[ref] Under that null hypothesis, the probability integral transform gives $U_i=F_0(X_i)$ with the uniform distribution on $(0,1)$, retaining independence. Consequently the sampling distribution of $D_n$ depends on the sample size $n$, but not on the particular continuous $F_0$. This is the useful sense in which the calibration is distribution-free.

For the exponential distribution of rate one, $F_0(x)=1-e^{-x}$ for $x\ge 0$. Take $x_i=-\log (1-u_i)$ for the four stipulated $u_i$ above. The probability integral transform changes the horizontal coordinates but preserves both the order statistics and their cumulative heights, so the same Kolmogorov-Smirnov statistic $3/10$ appears on both scales. More generally, for continuous $F_0$, the jump-side formulas depend only on $F_0(X_{(i)})$. Strict increase everywhere is unnecessary: intervals on which $F_0$ is constant have probability zero under the null hypothesis. This explains the invariance without assuming a strictly positive probability density function.

The observed Kolmogorov-Smirnov statistic is a discrepancy, whereas its p-value is a tail probability under the null hypothesis:Here $D_n^{\mathrm{new}}$ is computed from a fresh sample of the same sample size. Choose a significance level $\alpha$ before the test and reject when the calibrated p-value is at most $\alpha$. The value $D_4=3/10$ alone supplies neither a p-value nor a rejection decision. Failure to reject does not establish that the null hypothesis is true. A finite-sample calibration and the asymptotic Kolmogorov-Smirnov theorem are distinct; a cutoff for $D_n$ must be rescaled before use with $\sqrt{n}{D}_n$.^[ref]

Fitting statistical parameters on the tested observations replaces $F_0$ by the random $F_{\hat{\theta }}$. The resulting transformed observations need not be independent and identically distributed uniforms, so the ordinary reference law fails. The Lilliefors test addresses a particular fitted normal distribution. More general parametric bootstrap calibration must simulate from the fitted statistical model and repeat the fitting in every replicate; holding the fitted statistical parameters fixed during those comparisons calibrates a different procedure. Such a parametric bootstrap requires its own validity justification rather than inheriting exactness from the simple-null test.

For a discrete probability distribution, $F_0(X)$ is generally not uniform. Equal observations also arise from rounding, so tied data require a calibration that accounts for the discrete probability distribution or the rounding rule. The discrepancy remains meaningful, but continuous-null p-values are not automatically valid. Statistical dependence likewise changes the reference law. The two-sample Kolmogorov-Smirnov test compares two empirical distribution functions and has a different calibration.^[ref]

Nonregular identification at a flat mean map occurs when different parameter values give different probability distributions, yet the first derivative of the observable mean vanishes at a parameter of interest. Identifiability is a property of the entire parameter-to-distribution map; a first-order sensitivity calculation can miss it.

Consider an original statistical model with independent observationsThe cubic mean map is a strictly increasing function, so equality of the normal distributions implies equality of $\theta$. The model is globally identifiable, including at $\theta =0$. Nevertheless, the single-observation score function and Fisher information areThus $I(0)=0$ does not imply nonidentifiability. The rank is zero at the origin and one at every nonzero nearby point. This is precisely why a regular-point assumption matters: Rothenberg's local identification criterion assumes constant information-matrix rank in a neighborhood.^[ref]

Maximum likelihood estimation gives $\hat{\theta }=\mathrm{cbrt}(\overline{Y})$, using the real cube root. At the true value zero, write $\overline{Y}=Z/\sqrt{n}$ with a standard normal random variable $Z$. ThenThis is an exact sampling distribution identity for every $n$, not merely a convergence in distribution statement. The estimation error has scale $n^{-1/6}$ and a nonnormal shape. At a fixed nonzero $\theta$, the usual delta method instead gives asymptotic variance $1/(9n{\theta }^4)$; that approximation cannot be extended uniformly through zero.

An exact confidence interval through a cubic mean map follows by applying the strictly increasing function $\mathrm{cbrt}$ to both endpoints of the ordinary known-variance confidence interval for the mean:Here $z_p$ is the quantile function of the standard normal distribution. The transformed interval has exact coverage $1-\alpha$ for every $\theta$. When $n=64$ and the observed sample mean is zero, the 95% interval is approximately $[-0.626,0.626]$. Multiplying $n$ by 64 halves its width at that observed mean. A first-order interval based on dividing by the zero mean-map derivative would fail instead.

An order statistic is a value selected by its position in a sorted statistical sample. For real observations $X_1,\dots ,X_n$, writeThe $r$th order statistic $X_{(r)}$ is the $r$th smallest value, counting repeated values separately. The parentheses distinguish sorted position from the original observation label: $X_3$ and $X_{(3)}$ need not be equal. This definition permits ties and requires neither independence nor a continuous probability distribution. For a random statistical sample, each order statistic is itself a random variable and a statistic.

For example, the five fixed observationssort toTheir original indices in sorted order are $(2,5,4,1,3)$, so $x_{(3)}=x_4=0.4$. Their ranks of observations, listed in original observation order, are $(4,1,5,3,2)$. Here $x_{(1)}=0.1$ is the minimum, $x_{(5)}=0.9$ is the sample maximum, and $x_{(3)}=0.4$ is the sample median. These are deliberately chosen observations, rather than simulated draws.

Order statistics put extremes and typical values in one language: the first and last locate the minimum and sample maximum, while intermediate positions give sample quantiles and the sample median. Under the existing generalized-inverse convention, the sample quantile at level $p$ is $X_{(\lceil np\rceil )}$. For the five observations, the sample quantiles at $p=0.2,0.5,0.8$ are $0.1,0.4,0.7$. The distinction between a fixed observed value and its sampling distribution lets us quantify how much an extreme or sample median would vary across repeated statistical samples.

The useful counting identity isFor independent and identically distributed random variables with a continuous cumulative distribution function $F$, the count on the right has binomial distribution with parameters $n,F(t)$, givingThis cumulative distribution function is a binomial distribution tail: a question about sorting becomes a question about counting successes in Bernoulli trials. The counting proof also works for probability distributions with atoms and ties, although a probability density function need not then exist. For a uniform order statistic, $F(t)=t$ on $0\le t\le 1$, which makes the formula particularly explicit.

Let ${\mu }_0$ and ${\mu }_1$ be the expected political efficacy scores in the two employment groups in the target population. The null hypothesis is $H_0:{\mu }_0-{\mu }_1=0$ and the two-sided alternative hypothesis is $H_1:{\mu }_0-{\mu }_1\ne 0$. A Welch's t-test compares the difference of sample means with its standard error, estimating the two group variances separately:The reference Student t-distribution has approximate, generally noninteger statistical degrees of freedom; it does not require equality of the two variances. Observations should be independent, with finite group variances; normal distributions give the usual small-sample rationale, while large samples permit an approximate normal sampling distribution under suitable conditions. A large sample cannot correct a biased sampling procedure or dependence between observations.^[ref]

A p-value is a probability calculated under the null hypothesis and the sampling assumptions: for this two-sided test it measures the probability of an absolute test statistic at least as large as the observed one. It is not the probability that the null hypothesis is true, the probability that chance caused the observed difference, or a measure of the effect size. If the significance level $\alpha =0.05$ was chosen in advance, the printed $p=0.049521$ falls just below it. This would reject $H_0$ at 5%, but not at 1%. The significance level describes the long-run probability of a type I error, rejecting a true null hypothesis; a type II error instead fails to reject a false null hypothesis. Statistical power is the rejection probability under a specified alternative hypothesis. None of these errors for this particular realised study can be read as a posterior probability from $p$.

Both the printed sample means and the positive confidence interval point toward greater political efficacy in group 0, the non-employed group. Their magnitudes disagree. Subtracting the printed sample means gives $8.656650-6.322812=2.333838$ scale units. Yet the printed 95% confidence interval has midpoint $(0.2822355+0.6743412)/2=0.47828835$, not $2.333838$. A two-sided Welch interval is centred on the difference of sample means. This violates the consistency identities for a two-sided t-test; the interval and estimates cannot describe the same reported comparison. If the interval alone were trusted, it would exclude zero and indicate an increase between about $0.28$ and $0.67$ units, but that is explicitly a conditional interpretation. A 95% confidence interval refers to repeated-sample coverage of a fixed statistical parameter, not a 95% posterior probability for that realised interval.^[ref]

There are additional arithmetic warnings. The printed $t=6.1616$ at $2022.4$ statistical degrees of freedom gives a two-sided p-value about $8.67\times 10^{-10}$, not $0.049521$. Thus taking the test statistic as authoritative would imply much stronger evidence, not merely a marginal 5% result. The employment counts sum to $2311$, but their percentages are $905/2311\approx 39.16\%$ and $1406/2311\approx 60.84\%$, not $39.27\%$ and $60.73\%$. If those counts also apply to the two political efficacy sample means, their weighted arithmetic mean would be about $7.237$, not the reported overall $8.348$; separate complete-case group sizes are not supplied, so that last cross-output comparison is conditional on a common sample. No uniquely corrected difference, standard error or interval follows from these contradictory fields.

The political efficacy summary describes a bounded observed scale from $0$ to $16$, with median $9$ and middle quartiles $7$ and $12$. The interquartile range is $5$; the overall sample mean $8.348$ being below the median does not, by itself, establish a particular distributional shape. These descriptive statistics cannot supply the missing coherent group variances or a standardized effect size. Even a corrected small p-value would establish an association under a statistical model, rather than a causal effect of employment. Age, other background covariates, selection bias and the quality of the political efficacy measurement remain relevant. A defensible report gives the conditional 5% decision and direction, identifies the contradictions, and withholds a single numerical effect-and-uncertainty claim until the output is checked.

A null hypothesis specifies the population comparison to be tested, for example equality of the population means for women and their complement. The alternative hypothesis for a two-sided test is a nonzero difference. A Student's t-test measures an estimated difference in units of its estimated standard error:On the reported calibration, the sampling distribution under the null hypothesis is Student's t-distribution with $1225$ statistical degrees of freedom. Consequently the two-sided test has p-valueWith this many statistical degrees of freedom, the Student t-distribution is close to the standard normal distribution: $|t|=2.5$ exceeds the approximate $5\%$ two-sided critical value $1.96$, but falls short of the $1\%$ value $2.576$. Using the supplied calibration, reject at the $5\%$ significance level, but not at $1\%$. The large statistical degrees of freedom justify this comparison; a value $2.5$ need not be significant for an arbitrary very small number of statistical degrees of freedom.

The p-value is the probability, under the specified null hypothesis and sampling assumptions, of a test statistic at least as discrepant as the observed one. It is not the probability that the null hypothesis is true, nor the probability that the observed difference happened “by chance”. A significance level of $5\%$ controls the long-run type I error rate of the testing procedure when its assumptions hold. Failing to reject at $1\%$ does not prove equality. Statistical significance also does not measure the substantive size or importance of a difference.

The reported sample means place the women's mean $1.4$ scale units above the overall mean. The negative test statistic can reflect the order of subtraction; it does not reverse that observed ordering. An appropriate pre-specified directional one-sided test would have a tail p-value approximately $0.00627$ on the supplied calibration, but one cannot choose that direction after inspecting the results simply to halve a p-value.

There is a design issue behind the numerical calculation: the women are included in the overall sample. Comparing a subgroup mean with its containing sample mean requires accounting for shared observations. If $w$ is the women's sample fraction and ${\overline{x}}_O$ the mean for the remaining respondents, thenThese two estimates are not independent. An ordinary independent-two-sample Student's t-test of women against the entire respondent sample would therefore use an inappropriate standard error. The interpretation above accepts the supplied test statistic and its stated null calibration; a valid analysis of the underlying data should compare disjoint groups, or calculate the shared-sample covariance correctly. The summary alone cannot establish which calculation was used. Simple random sampling supports sampling inference under the stated design, but an observed gender association does not establish a causal effect.

An ancillary statistic has a sampling distribution independent of the unknown parameter. It describes a realized aspect of the sample configuration without supplying a marginal likelihood for the parameter. Its observed value can nevertheless be crucial to the precision of inference from the rest of the data. In a transitive transformation model, invariance gives a common distribution under every parameter value, so maximal invariants provide natural ancillaries, such as the standardized configuration in a location-scale family.

Conditional inference uses the sampling distribution of an estimator or sufficient statistic conditional on the observed ancillary. With joint coordinates $(T,A)$, the factorizationshows that the conditional likelihood and the joint likelihood differ only by a parameter-free data factor. Conditional reference distributions, confidence intervals and p-values can still differ markedly from unconditional ones, because they adapt to the realized configuration. For a continuous ancillary one uses a regular conditional distribution or conditional density; dividing by the zero probability of a single ancillary value is not a valid definition.

An exact example makes this adaptation visible. Let independent observations be uniform on $(\theta -1/2,\theta +1/2)$, with $n\ge 2$. Put $U=X_{(1)}$, $V=X_{(n)}$, $R=V-U$ and $C=(U+V)/2$. Specifying the minimum and maximum contributes $n(n-1)$ choices of observations, while each remaining observation must lie between them. HenceThe transformation $u=c-r/2$, $v=c+r/2$ has unit absolute Jacobian determinant. Its support is $0<r<1$, $|c-\theta |<(1-r)/2$. Integrating over $c$ giveswhich is independent of $\theta$, proving that the range is ancillary. Dividing the joint density by this marginal density givesThe centered conditional pivot is uniform, sohas conditional coverage $1-\alpha$ at every $0<r<1$, and hence unconditional coverage $1-\alpha$ as well. A large realized range leaves little room for translating the interval and gives more precise inference. This is range-conditioned uniform location inference. The range and center are not independent: the conditional spread depends on the range, even though the range is ancillary.

Completeness can give independence in other models. Suppose $S$ is a complete sufficient statistic and $A$ is ancillary. For any bounded measurable $h$, sufficiency makes $g(S)=E_{\theta }(h(A)\mid S)$ a function independent of $\theta$, while ancillarity makes $c=E_{\theta }h(A)$ independent of $\theta$. Since $E_{\theta }(g(S)-c)=0$ for every $\theta$, completeness implies $g(S)=c$ almost surely. Taking all bounded indicator functions $h$ proves independence of $A$ and $S$. This is the content of Basu's theorem, with the proof displaying precisely where sufficiency and completeness are used. Without completeness the conclusion fails: in the uniform location example the minimum and maximum are sufficient, and contain the nondegenerate ancillary range.

There are practical qualifications. Ancillaries need not be unique, and some models supply only approximate ancillary coordinates; one must specify the conditioning surface and the accuracy intended. Conditioning on a statistic sufficient for a nuisance parameter is a different device: it may remove that nuisance even though the statistic is not ancillary for the full parameter. Conditioning indiscriminately can discard useful information. In higher-order likelihood inference, the ancillary coordinates specify how the fitted data are varied, entering both the p* density and the sample derivatives in the modified profile likelihood. Their role is therefore substantive, rather than merely a label for an extra statistic.

With the divisor-$n$ sample covariance matrix defined above, the maximum-likelihood estimators areTheir joint sampling distribution is specified byHere $W_p(\nu ,V)$ uses the Wishart distribution convention $\mathbb{E}W=\nu V$. Thus the covariance maximum-likelihood estimator has expected value $(n-1)V/n$; the unbiased estimator of the covariance matrix is $nS/(n-1)$. These distributions follow from orthogonal projections of the jointly multivariate normal observations onto the constant-observation direction and its orthogonal complement, though no proof is required here. The covariance maximum in the positive-definite parameter space requires $S$ nonsingular, which holds almost surely when $n>p$ and $V$ is positive definite. Otherwise the rank of a centered sample covariance matrix shows why the usual unrestricted finite maximum does not exist.

For the initial normal linear model, the design matrix has rows $(1,x_1,x_2,x_3)$. The eight factorial corners have balanced signs and orthogonal coordinate columns; the six centre points contribute only to the intercept. ConsequentlyUsing the least-squares normal equations gives the least-squares estimatorIt is a linear transformation of independent normal responses, so its sampling distribution isThe four estimated coefficients are independent, since they are jointly normal with diagonal covariance matrix.

Adding the axial settings produces a central composite design: eight factorial corners, six axial points and six replicated centre points. The axial distance $1.682$ is the rounded value of $8^{1/4}=1.6817928\dots$. With that exact distance it is a rotatable design for a full quadratic model. Indeed, sign symmetry annihilates odd moments, all coordinates have the same second moment, and the fourth moments satisfyRotational symmetry requires the first to be three times the second, giving ${\alpha }^4=8$. Using the printed rounded distance makes rotatability approximate, rather than mathematically exact.

The full quadratic regression mean has ten coefficients:The expanded design matrix has rank ten. To verify this rather than just count parameters, suppose such a quadratic vanishes at every setting. Its centre value sets its constant term to zero; subtracting the values at opposite axial points sets every linear coefficient to zero, and adding them then sets every squared-coordinate coefficient to zero. On the factorial corners, orthogonality of the three pair-product columns forces their coefficients to zero as well. Thus all ten columns are independent, leaving $20-10=10$ residual statistical degrees of freedom.

There are $15$ distinct settings: eight corners, six axial points and the centre. The six centre replicates give the only pure error, with $6-1=5$ degrees and sum of squares $859.33$. The saturated setting-mean model has $15$ parameters, so the lack of fit beyond the quadratic model has $15-10=5$ degrees. For any setting $r$ with $n_r$ replicates, the identitysplits the residual sum of squares into pure error and lack of fit; the cross term vanishes because the deviations from the setting mean sum to zero. Summing givesUnder a correct quadratic mean with independent errors of constant normal variance, the pure-error and lack-of-fit projections are orthogonal, hence their sums of squares are independent with respective distributions ${\sigma }^2{\chi }_5^2$ and ${\sigma }^2{\chi }_5^2$. The Lack-of-fit F-test therefore usesThis is below the given $10\%$ and $5\%$ critical values $3.45$ and $5.05$. There is no significant lack of fit at either level. That is failure to reject this quadratic mean, not proof that it is exact or that a maximum has already been located. The pure-error denominator also assumes the error variance at the centre represents the common variance throughout the experiment.

The incremental cost-effectiveness ratio must be read with the signs of both increments. A negative ratio can mean either greater effectiveness at lower cost, which is favorable, or lower effectiveness at greater cost, which is unfavorable. A positive ratio can likewise lie in either the more-costly/more-effective or cheaper/less-effective quadrant. Thus ordering ratio values without locating the cost-effectiveness plane quadrant can reverse a decision.

The incremental cost-effectiveness ratio is undefined at a zero effect increment and extremely unstable near it. Its sampling distribution can be highly skewed and its mean may fail to exist; a usual symmetric confidence interval or an interval spanning positive and negative ratios can conceal that instability. Cost-effect correlation must be retained, as the Fieller confidence set calculation illustrates.

Interpretation also requires a specified comparator, effect unit, costing perspective, follow-up horizon and willingness-to-pay threshold. Different comparator choices give different incremental ratios, and lower ratios alone do not settle decisions across differently scaled outcomes. Longer survival need not mean a proportional quality-of-life gain; extrapolation, censored outcomes and omitted costs can affect the increments. Point estimates conceal both sampling uncertainty and uncertainty in these assumptions.

For a positive effect increment, comparison with a threshold $\lambda$ is equivalent to $\Delta C/\Delta E<\lambda$. For negative effect increments that inequality reverses when multiplied by $\Delta E$, so a rule that blindly compares all ratios with $\lambda$ is wrong. Use the signs and joint cost/effect uncertainty, or the incremental net monetary benefit $\lambda \Delta E-\Delta C$, rather than the ratio alone. No particular threshold is given in the paper, so the point estimate does not establish whether reimbursement should occur.

Minimizing the residual sum of squares $\Vert Y-Xb{\Vert }^2$ gives the least-squares normal equations $X^{T}X\hat{\beta }=X^{T}Y$. Since the design matrix has full column rank, $X^{T}X$ has a matrix inverse, soThe sampling distribution follows because a linear transformation of a multivariate normal distribution is again a multivariate normal distribution. Its expectation is $\beta$, and its covariance matrix is ${\sigma }^2(X^{T}X{)}^{-1}{X}^{T}X(X^{T}X{)}^{-1}={\sigma }^2(X^{T}X{)}^{-1}$.

The hat matrix is $H=X(X^{T}X{)}^{-1}{X}^T$. It is the orthogonal projection matrix onto the column space of $X$: $H^T=H$, $H^2=H$ and $HX=X$. Thus the regression residuals are $\hat{\epsilon }=(I-H)Y$, withThis is fitted-residual orthogonality. The vectors are jointly normal, so independence of uncorrelated jointly normal variables strengthens both zero-covariance statements to independence.

For the quadratic linear regression, the columns of the design matrix are $1,x,x^2$. The replicated design givesFor $v(x)=(1,x,x^2{)}^T$, the variance of a fitted regression mean is ${\sigma }^{2}v(x{)}^T(X^{T}X{)}^{-1}v(x)$, henceWriting $u=x^2\in [0,1]$, this is the quadratic $({\sigma }^2/20)(2-3u+3u^2)$. Its derivative vanishes at $u=1/2$; it decreases before that point and increases afterwards. Therefore the maximum is ${\sigma }^2/10$ at $x=-1,0,1$, and the minimum is ${\sigma }^2/16$ at $x=\pm 1/\sqrt{2}$.

At $100^{\circ }\mathrm{C}$, $x=0$ and the expected yield is $\alpha$. The unbiased residual variance estimate is $s^2=2.43/(30-3)=0.09$. By Cochran's theorem, $27s^2/{\sigma }^2\sim {\chi }_{27}^2$ independently of $\hat{\alpha }$, giving the Student t confidence intervalThe interval estimates the expected yield, so it uses the variance of a fitted regression mean without an additional future-observation error term. The supplied summary determines the half-width but does not give a numerical value of $\hat{\alpha }$.

Write $q=n-p$. In this normal linear model, the ordinary least squares criterion is $\Vert Y-Xb{\Vert }^2$. Differentiating with respect to $b$ gives the normal equations for linear least squares, $X^{T}Xb=X^{T}Y$. Full column rank of the design matrix makes $X^{T}X$ invertible, and completing the square proves that the unique minimum isSince $\hat{\beta }=\beta +(X^{T}X{)}^{-1}{X}^Tϵ$, its sampling distribution is the multivariate normal distributionLet $H=X(X^{T}X{)}^{-1}{X}^T$ be the hat matrix. It is symmetric and idempotent with rank $p$, and $HX=X$. The regression residual vector is $(I-H)Y=(I-H)ϵ$, soAn orthogonal projection of a Gaussian vector onto the residual space gives $\mathrm{RSS}/{\sigma }^2\sim {\chi }_q^2$. The residual projection and fitted projection have zero cross-covariance, hence are independent because they are jointly normal. In particular, $s^2$ is an unbiased estimator of ${\sigma }^2$ and is independent of $\hat{\beta }$. The maximum-likelihood estimator of ${\sigma }^2$ instead divides by $n$ and is not unbiased.

For the null hypothesis ${\beta }_1=0$, put $v_{11}=((X^{T}X{)}^{-1}{)}_{11}$. Under that null hypothesis, ${\hat{\beta }}_1/(\sigma \sqrt{v_{11}})$ is standard normal and independent of $q{s}^2/{\sigma }^2$. Therefore the Student t-test usesA two-sided level-$\alpha$ test rejects when $|T|>t_{q,1-\alpha /2}$. The denominator is the estimated standard error of the regression coefficient.

The fitted linear regression includes an intercept and three covariates:Thus $n=25$, $p=4$ and the residual degrees of freedom are $21$. The fitted conditional mean isEach slope measures a conditional association with the response while the other two covariates are held fixed. Increasing wind speed by one mile per hour corresponds to a fitted decrease of $0.527410$ in the concentration units used. Its standard error is $0.224904$, giving $t=-2.345$ and a two-sided p-value of $0.0289$: evidence of a negative conditional association at the 5% level. The temperature slope is $0.124991$ per degree Fahrenheit, with standard error $0.075173$, $t=1.663$ and p-value $0.1112$. The insolation slope is $-0.005259$ per langley per day, with standard error $0.006637$, $t=-0.792$ and p-value $0.4370$. Neither of these last two conditional effects is significant at 5%; that does not establish that either true slope is zero. These are associations, not a demonstration of causal effects.

The intercept estimates the mean at zero values of all three covariates. Its value $3.784916$ has a large standard error $7.979647$, giving $t=0.474$ and p-value $0.6402$. The zero-covariate combination may be outside the observed range, so this test may have little practical relevance. Every coefficient test uses a Student t-distribution with $21$ degrees of freedom.

The reported residual standard deviation $s=1.844$ measures unexplained variation in concentration units; the subsequent table gives $\mathrm{RSS}=71.428$, consistent with $s=\sqrt{71.428/21}$ after rounding. The five-number residual summary describes the spread, from $-2.3052$ to $3.4033$, with median $-0.4990$. An intercept forces the residual sum to zero, not the residual median. These five numbers alone cannot establish normal errors, constant variance or independence; residual-versus-fitted and quantile-quantile plots would be more informative, and successive days also warrant checking serial dependence. The coefficient of determination $R^2=0.6533$ means that 65.33% of the sample response sum of squares about its mean is explained by this fit. Its adjusted version iswhich accounts for the number of estimated regression coefficients.

The stepwise selection by the Akaike information criterion searches between the intercept-only model and the specified model with all three main effects. For these Gaussian fits, the printed Akaike information criterion is, up to a constant common to the candidates,The full model has $p=4$ and AIC $34.246$. Removing insolation increases the residual sum of squares by $2.136$ but decreases AIC to $32.982$, so that removal is chosen. Removing temperature or wind instead would give AIC $35.337$ or $38.060$ and is worse. In the next step the candidate models include deletions and restoration of insolation. Keeping wind and temperature has the smallest AIC: removing temperature gives $33.389$, restoring insolation gives $34.246$, and removing wind gives $37.960$. Hence the search stops, givingThe accompanying F-tests compare a larger model with a one-parameter reduction: $F=\Delta \mathrm{RSS}/s^2$, where $s^2$ is estimated from the larger model. For example, removing insolation gives $2.136/(71.428/21)=0.628$, equal to the corresponding squared test statistic. Deletions from the second-stage model use $73.564/22$ in the denominator, whereas restoring insolation uses the full-model denominator. These tests describe the comparisons; they do not determine the AIC choice. In particular, temperature is retained despite its deletion-test p-value $0.15016$. AIC balances fit and complexity, and a stepwise optimum need not be a global optimum over a richer model class. Ordinary coefficient inference after choosing a model does not account for the selection itself.

For positive responses, the Box–Cox transformation is $T_{\lambda }(y)=(y^{\lambda }-1)/\lambda$ for $\lambda \ne 0$, and $T_0(y)=\log y$. Its profile likelihood refits the linear regression and variance at each $\lambda$ and includes the transformation Jacobian. The graph peaks at roughly $\lambda =0.3$–$0.4$. The approximate 95% likelihood-ratio confidence interval contains values a little below zero through approximately one, using the cutoff ${\ell }_{\max }-{\chi }_{1,0.95}^2/2$. Thus a logarithmic transformation ($\lambda =0$) and a square-root transformation ($\lambda =1/2$) are plausible, while leaving the response on its original scale ($\lambda =1$) is at or very near the upper confidence boundary. The broad profile supports considering a mild power transformation; it does not determine a sharply estimated power. The raster does not justify more precise endpoints or a decisive assertion about which side of the boundary $\lambda =1$ lies.

Put $G=X^{T}X$ and $H=X{G}^{-1}{X}^T$. The rank of a matrix assumption makes $G$ invertible, and $H$ is the orthogonal projection matrix onto the column space of the design matrix. The Gaussian likelihood has log-likelihoodMinimizing the squared Euclidean norm gives the ordinary least squares normal equations $G\hat{\beta }=X^{T}Y$. Maximizing over $v$ then gives the maximum-likelihood estimatorsThe divisor for the maximum-likelihood estimator of the variance is $n$, whereas the unbiased residual variance estimate is $s^2=\mathrm{RSS}/(n-p)$. The exceptional event $\mathrm{RSS}=0$ has probability zero under the Gaussian linear model with ${\sigma }^2>0$; on that event the likelihood is unbounded as $v\downarrow 0$.

For the joint sampling distribution, write $\hat{\beta }-\beta =G^{-1}{X}^Tϵ$ and $Y-X\hat{\beta }=(I-H)ϵ$. Their cross-covariance matrix vanishes because $X^T(I-H)=0$. These two vectors are jointly multivariate normal, so they are independent random variables. The residual orthogonal projection has rank $n-p$, hence the standard chi-squared distribution result givesThus these two displayed marginal laws, with their independence, specify the full joint distribution.

The quadratic formhas a chi-squared distribution with $p$ degrees of freedom and is independent of $\mathrm{RSS}/{\sigma }^2$. Their ratio therefore supplies the exact F-distribution pivotIf $f_{1-\alpha }$ is the $(1-\alpha )$ quantile of that F-distribution, inversion gives the normal linear-model confidence ellipsoidFor every $\beta$ and every ${\sigma }^2>0$, the probability that this confidence set contains $\beta$ is exactly $1-\alpha$. To test the null hypothesis, reject precisely when ${\beta }_0$ lies outside this confidence ellipsoid, equivalently when $T({\beta }_0)>f_{1-\alpha }$. Under the null hypothesis the rejection probability is exactly $\alpha$, independent of the unknown variance.

For the numerical calculation, the second column of the design matrix sums to zero and its sum of squares is $60$. Direct calculation from the observations givesConsequently the ordinary least squares estimate, residual sum of squares and unbiased residual variance areThe maximum-likelihood estimator of the variance is instead $7/9$. At the proposed null value the F-distribution statistic isTherefore reject the null hypothesis at the 5% level. The relevant critical value has $2$ numerator and $7$ residual degrees of freedom.

Group means are independent random variables with normal distributionsThe resulting univariate sampling distributions areThe same variance calculation holds for every $i\ge 2$, since the difference involves two independent group means. Therefore the standard errors satisfyReplacing $\sigma$ by a common estimated error standard deviation preserves this ratio. Although the individual group means are independent, the contrasts ${\hat{\alpha }}_i$ share the baseline mean and have covariance ${\sigma }^2/J$ for distinct $i\ge 2$.