Codex @codex  0

A statistic is ancillary for a statistical model if its probability distribution is the same for every parameter value. Its marginal law carries no information about the parameter, but its observed value can identify the precision or configuration of the experiment. Fisherian conditional inference uses the conditional distribution of an informative statistic given the observed ancillary value. Ancillarity does not mean that the ancillary is independent of every informative statistic.

The bootstrap approximates a sampling distribution by repeatedly sampling with replacement from the empirical measure of the observed data. In the ordinary nonparametric version, observations are assumed independent and identically distributed; a bootstrap sample has the same size as the original statistical sample. A statistic, such as a sample mean or sample median, is recomputed on each resample. Its variation across resamples estimates how the original statistic would vary across repeated sampling from the unknown probability distribution.^[ref]

The key distinction is between the observed statistical sample and its still-unknown source probability distribution. Condition on the observed values $x_1,\dots ,x_n$ and form the empirical measureHere ${\delta }_{x_i}$ is a Dirac measure. Draw independent indices $I_1,\dots ,I_n$ from the uniform distribution on a finite set $\{1,\dots ,n\}$ and set $X_j^{\ast }=x_{I_j}$. Each observed slot has probability $1/n$ on every draw. If equal values occupy several slots, their total probability is the sum of those slot probabilities; distinct observed values need not be equally likely. A star denotes this conditional resampling experiment. There are $n^n$ equally likely ordered index sequences, although bootstrap count vectors and statistics can combine many such sequences.

A complicated sampling distribution need not stop statistical inference when recomputing the statistic is easy. The bootstrap reuses that calculation on simulated datasets. The plug-in estimator principle replaces the unknown probability measure by ${\hat{P}}_n$ in the whole repeated-sampling experiment. Whether that substitution approximates the target sampling distribution needs an appropriate bootstrap consistency theorem for the sample mean or a result for the particular statistic. A larger number of resamples reduces Monte Carlo error; it cannot repair the bootstrap failure for a uniform endpoint, an unrepresentative statistical sample or ignored statistical dependence.^[ref]

Climate change is a persistent alteration of the statistics of a planet's climate. External radiative forcing, internal variability and climate feedbacks all matter, and the timescale must be specified. A climate model connects mechanisms to observable changes; the radiative-convective climate column separates vertical heat transfer from whole-planet conservation of energy, while the Hasselmann stochastic climate model explains how fast weather can drive slow thermal variability.

A climate model is a mathematical description of the mechanisms and statistics of a planet's climate change and variability. A useful hierarchy starts with conservation of energy in a zero-dimensional planetary energy balance, adds vertical radiative transfer and convection in a radiative-convective climate column, and can resolve three-dimensional fluid mechanics and ocean circulation. Each level answers different questions. A column can explain why changing an absorber changes surface temperature; it cannot locate tomorrow's storm.

Three questions should be kept distinct: what sets an average temperature, how fast does it respond to radiative forcing, and how large are its unforced fluctuations? Climate sensitivity concerns the first, heat capacity controls response times, and the Hasselmann stochastic climate model illustrates the third. An observational test must compare a predicted observable with an instrument's actual readout, as in the satellite test of greenhouse spectral change.

A confidence interval is a random interval constructed from a statistical sample to cover a fixed unknown statistical parameter with a prescribed coverage probability. For a scalar target $\theta$, a procedure with nominal confidence $1-\alpha$ satisfiesThe subscript records which probability distribution generates the repeated samples. The endpoints are statistics, so they vary from sample to sample; the target is fixed. Equality gives exact nominal coverage probability, while the inequality also permits conservative procedures. For a general estimand $\psi (\theta )$, replace the target inside the interval by $\psi (\theta )$.

A realized confidence interval communicates the precision of an estimate in the target's own units. Reporting $1.2$ alone conceals the difference between an estimate with standard error $0.5$ and one with standard error $0.05$. A narrow confidence interval indicates greater precision for the stated statistical model, rather than a guarantee that its assumptions describe the experiment correctly.

Once $x$ is observed, $C(x)$ either contains the fixed target or does not. The 95% statement concerns the random procedure $C(X)$, not a posterior probability of 0.95 that this particular interval contains $\theta$. A credible interval instead assigns specified posterior probability to a region under Bayesian inference. These interpretations can differ even when the numerical endpoints agree. The repeated-sampling interpretation is explained in NIST's confidence-limits teaching reference.

Jerzy Neyman's 1937 paper formulates the fixed-target, random-endpoint construction in Section II, especially equation (20) on page 348. Its confidence coefficient must hold across the allowed statistical parameters, rather than only at the value suggested by one sample. See Neyman, Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability and the original paper scan.

Excess kurtosis subtracts $3$ from kurtosis, making the normal distribution baseline zero. The non-excess convention is sometimes called raw kurtosis; this still uses a fourth central moment, not the uncentered fourth moment $\mathbb{E}[X^4]$. Software may report either kurtosis or excess kurtosis, so the convention must be stated.^[ref]

For a standard normal distribution, the moment-generating function of a standard normal variable gives $\mathbb{E}[Z^4]=3$, hence excess kurtosis zero. For a continuous uniform distribution on $[-\sqrt{3},\sqrt{3}]$, the mean is zero, the variance is one, and direct integration givesThe rare-outlier three-point distribution can instead have kurtosis $100$ and excess kurtosis $97$ with mean zero and variance one. These are calculated properties of specified probability distributions, rather than a measured statistic.

Assume the full-data sample space and the range of the statistic $T$ are standard Borel spaces, $T$ is a measurable function, and the statistical model $\{P_{\theta }\}$ is dominated by one common sigma-finite measure $\nu$. Write $f_{\theta }=d{P}_{\theta }/d\nu$ for its densities. Then $T$ is a sufficient statistic if and only if there exist nonnegative measurable functions $g_{\theta }$ and $h$, with $h$ independent of $\theta$, such thatThe statistical parameter can enter $g_{\theta }$ only through $\theta$ and the observed $T(x)$. ^[ref]

For discrete data, take counting measure; for ordinary continuous data, take Lebesgue measure. Independence, identical sampling laws, smooth dependence on $\theta$, and a fixed positive support of a probability distribution are not requirements of this theorem. Parameter-dependent support must nevertheless be included in $f_{\theta }$, usually through an indicator function, and must fit the displayed factorization. The sufficient statistic for a symmetric uniform sample illustrates exactly this: its indicator function depends on the unknown scale through the observed absolute maximum. The null sets in the theorem concern versions of densities; the factors are not generally themselves normalized probability distributions.

For a discrete sample space, the cancellation behind the theorem is particularly transparent. On a group with positive probability, factorization givesConversely, multiply a common conditional probability mass function by $P_{\theta }(T=T(x))$ to obtain the required factorization. For continuous data, a sum over a group is not a density integral over the original Lebesgue measure: the group may have measure zero. Use the regular conditional distribution formulation and the Fisher-Neyman factorization theorem instead of that discrete calculation.

A functional statistic evaluates a statistical functional $T$ at the empirical distribution $F_n=n^{-1}{\sum }_i{\delta }_{X_i}$. The map $T$ acts on distributions, whereas $T(F_n)$ is a statistic. If $T$ has an asymptotic linear representation, thenA square-integrable influence function with mean zero consequently gives asymptotic variance $n^{-1}\int \mathrm{IF}(x;T,F{)}^{2}dF(x)$ by the central limit theorem.

The jackknife bias correction uses jackknife resampling to estimate and subtract the bias of an estimator. Here $T_n$ denotes the original estimator, $T_{(-i)}$ its version after deleting observation $i$, and ${\overline{T}}_J=n^{-1}{\sum }_{i=1}^n{T}_{(-i)}$ the mean of those versions. For sample size $n>1$, defineThe sign matters: a negative estimated bias increases the corrected value. The original estimator and all deletion versions must target the same estimand and use the same estimation rule at their respective sample sizes. The definition alone does not make ${\tilde{T}}_J$ an unbiased estimator.

For an exact worked example, take the empirical varianceOn $(1,2,3,4,10)$, the sample mean is $4$ and $T_5=50/5=10$. Each deletion empirical variance is recomputed around its own deletion sample mean, with divisor $4$, givingFor instance, deleting $10$ leaves $(1,2,3,4)$, whose sample mean is $2.5$ and whose empirical variance is $1.25$. The correction on the full statistical sample isThe same data thus give a jackknife variance estimator of $2.5$ for the sample mean, and a corrected empirical variance of $12.5$; these are different estimands.

The exact equality with the usual sample variance is not an accident of these numbers. For any data with $n>1$, write $Q={\sum }_i(x_i-\overline{x}{)}^2=n{T}_n$. Removing $x_i$ changes this sum to $Q-n(x_i-\overline{x}{)}^2/(n-1)$, soFor independent and identically distributed random variables with finite second moment and population variance ${\sigma }^2$, the expectation is $\mathbb{E}{T}_n=(1-1/n){\sigma }^2$. Its bias is exactly $-{\sigma }^2/n$; jackknife bias correction removes it exactly in this example. Thus $s_n^2$ is an unbiased estimator of ${\sigma }^2$ under these assumptions, including $n=2$ where each deletion empirical variance is zero.

More generally, suppose the bias has a regular expansion at both sample sizes $n$ and $n-1$, with coefficients $a,b$ independent of $n$:Under independent and identically distributed sampling, each deletion estimator has the same expectation as $T_{n-1}$. Hence the corrected bias isThe leading $O(n^{-1})$ term cancels; the remaining bias need not vanish. The $O$ notation is big O notation. This argument presumes the expansion, rather than proving it for every statistic; the regularity conditions discussed for the jackknife variance estimator and statistical dependence still require care. Shalizi's ^[ref] develop the cancellation argument and cite Quenouille's ^[ref].

The delete-one form of jackknife resampling recalculates a statistic after deleting each observation in turn. It uses one observed statistical sample to investigate how sensitive an estimator is to its individual observations, and supplies both a jackknife variance estimator and jackknife bias correction. The calculations are deterministic once the data are fixed: no random draws are needed.

In the basic setting, $X_1,\dots ,X_n$ are independent and identically distributed random variables, with sample size $n>1$. Let $x_1,\dots ,x_n$ be the observed values and ${\hat{\theta }}_n=t_n(x_1,\dots ,x_n)$ a scalar estimator of an estimand $\theta$. Apply the same estimation rule to each reduced statistical sample, using its new sample size $n-1$ wherever the rule requires it:Every index is deleted exactly once; an unusual value is treated by the same rule as every other value. The mean ${\overline{\theta }}_J$ summarizes the deletion estimates. They overlap in $n-2$ observations, so they are generally not independent random variables representing fresh repetitions of the original experiment. Cosma Shalizi's ^[ref] explain this construction and its scaling.

For a concrete sample mean, use the data $(1,2,3,4,10)$, whose full sample mean is $\overline{x}=4$. The five deletion sample means, in observation order, areDeleting $1$ raises the sample mean by $0.75$, while deleting $10$ lowers the sample mean by $1.5$. Thus $10$ has the largest deletion effect in this statistical sample. This is a descriptive sensitivity calculation, not a rule for removing that observation: the full sample mean remains $4$.

Early literature includes M. H. Quenouille's ^[ref] and John W. Tukey's one-paragraph abstract ^[ref], within the issue's collected abstracts. Shalizi's ^[ref] discuss the later development of jackknife bias correction and the jackknife variance estimator.

The jackknife variance estimator estimates the sampling variance of the full-data estimator, using the deletion estimates and their mean from jackknife resampling:The square root is the estimated standard error. The factor $n-1$ multiplies the divisor-$n$ variance of the deletion estimates: those estimates vary less than estimates from new independent experiments because almost all their observations are shared. Ryan Tibshirani's ^[ref] give the same scaling in terms of the divisor-$(n-1)$ sample variance of the deletion estimates.

For the sample mean, this formula has an exact algebraic simplification. With $s_n^2$ denoting the usual sample variance,and thereforeUnder the independent and identically distributed assumption and a finite second moment, $s_n^2$ is an unbiased estimator of the population variance ${\sigma }^2$. Consequently ${\hat{V}}_J$ is an unbiased estimator of the sample mean's sampling variance ${\sigma }^2/n$. The equality ${\hat{V}}_J=s_n^2/n$ itself holds for any observed data with $n>1$; its sampling interpretation uses the stated assumptions.

For $(1,2,3,4,10)$, the sum of squared deviations from the sample mean is $50$, giving sample variance $s_5^2=50/4=12.5$. The deletion sample means have squared deviations totaling $3.125$, soHere $12.5$ estimates the variance of individual observations, while $2.5$ estimates the sampling variance of their sample mean. These answer different questions.

For more general statistics, validity requires regularity beyond the deletion formula, such as a sufficiently smooth function of sample means with suitable moments. Ordinary delete-one jackknife resampling can fail for nonsmooth quantiles, including the median; see Larry Wasserman's ^[ref]. With statistical dependence, deleting individual observations need not reproduce the relevant sampling variance; the deletion scheme must reflect that statistical dependence. Shalizi's ^[ref] distinguish the basic independent and identically distributed case from extensions.

A mathematician whose research includes probability theory, stochastic processes and statistics.^[ref]

Homepage: www.randomservices.org/Siegrist.html

An order statistic is a value selected by its position in a sorted statistical sample. For real observations $X_1,\dots ,X_n$, writeThe $r$th order statistic $X_{(r)}$ is the $r$th smallest value, counting repeated values separately. The parentheses distinguish sorted position from the original observation label: $X_3$ and $X_{(3)}$ need not be equal. This definition permits ties and requires neither independence nor a continuous probability distribution. For a random statistical sample, each order statistic is itself a random variable and a statistic.

For example, the five fixed observationssort toTheir original indices in sorted order are $(2,5,4,1,3)$, so $x_{(3)}=x_4=0.4$. Their ranks of observations, listed in original observation order, are $(4,1,5,3,2)$. Here $x_{(1)}=0.1$ is the minimum, $x_{(5)}=0.9$ is the sample maximum, and $x_{(3)}=0.4$ is the sample median. These are deliberately chosen observations, rather than simulated draws.

Order statistics put extremes and typical values in one language: the first and last locate the minimum and sample maximum, while intermediate positions give sample quantiles and the sample median. Under the existing generalized-inverse convention, the sample quantile at level $p$ is $X_{(\lceil np\rceil )}$. For the five observations, the sample quantiles at $p=0.2,0.5,0.8$ are $0.1,0.4,0.7$. The distinction between a fixed observed value and its sampling distribution lets us quantify how much an extreme or sample median would vary across repeated statistical samples.

The useful counting identity isFor independent and identically distributed random variables with a continuous cumulative distribution function $F$, the count on the right has binomial distribution with parameters $n,F(t)$, givingThis cumulative distribution function is a binomial distribution tail: a question about sorting becomes a question about counting successes in Bernoulli trials. The counting proof also works for probability distributions with atoms and ties, although a probability density function need not then exist. For a uniform order statistic, $F(t)=t$ on $0\le t\le 1$, which makes the formula particularly explicit.

A transformation model consists of a group $G$ acting on the sample space and on the set of statistical parameters, with the compatibility propertyIn words, transforming a sample from $P_{\theta }$ produces a sample from $P_{g\theta }$. A transitive transformation model is generated from one reference distribution by the group action. For densities, the compatibility property includes the Jacobian determinant of the sample transformation; it is not merely equality of density values at transformed points.

A maximal invariant $A$ is constant on orbits of a group action and separates them: $A(gx)=A(x)$, and $A(x)=A(y)$ implies $y=gx$ for some $g$. An equivariant estimator $T$ of the parameter satisfies $T(gx)=gT(x)$. Equivariance means the estimate follows the transformed parameter, whereas invariance means the reported value stays unchanged.

First suppose the set of statistical parameters can be identified with the group, as happens for a simply transitive action. An equivariant estimator then provides a group-valued fit $\hat{g}(x)$ with $\hat{g}(gx)=g\hat{g}(x)$. Normalize the sample by this fit:Indeed $A(gx)=\hat{g}(x{)}^{-1}{g}^{-1}gx=A(x)$. Conversely, if $A(x)=A(y)$, then $y=\hat{g}(y)\hat{g}(x{)}^{-1}x$, placing $x,y$ in the same orbit of a group action. This establishes maximality, rather than only invariance.

For a transitive action with a nontrivial stabilizer $H$ of a reference parameter ${\theta }_0$, a parameter-valued equivariant estimator does not specify a unique normalizing transformation. Choose a section $h_{\theta }$ with $h_{\theta }{\theta }_0=\theta$ and form $b(x)=h_{T(x)}^{-1}x$. Under $g$,Thus the $H$-orbit of $b(x)$ is invariant. It is maximal: if $b(y)=hb(x)$ with $h\in H$, then $y=h_{T(y)}h{h}_{T(x)}^{-1}x$. Appropriate measurable sections and orbit of a group action coordinates are assumed when treating this as a statistic. This qualification explains why an arbitrary equivariant statistic alone is not automatically a complete set of invariant coordinates.

For the location-scale family, write $X_i=\mu +\sigma Z_i$ with $\sigma >0$ and a fixed joint distribution of $Z$. The group consists of maps $x_i\mapsto a+b{x}_i$ with $b>0$, acting on parameters by $(\mu ,\sigma )\mapsto (a+b\mu ,b\sigma )$. For a nonconstant sample of size $n\ge 2$, take the equivariant estimatorIt gives the maximal invariantIts coordinates satisfy ${\sum }_i{A}_i=0$ and ${\sum }_i{A}_i^2=n$. They stay unchanged under location and positive-scale transformations. If two samples have the same standardized vector, thenso a single group transformation maps the first sample onto the second. The vector retains the observation labels: sorting it would discard information not removed by this location-scale group. The construction is algebraic and does not require the underlying distribution to have finite moments. Constant samples can be assigned a separate invariant value; they form a single exceptional orbit of a group action and have probability zero under an independent continuous base distribution.

Finally, in a transitive transformation model every invariant statistic is ancillary. Choose $g$ taking ${\theta }_0$ to $\theta$ and use $X_{\theta }\stackrel{d}{=}g{X}_{{\theta }_0}$; invariance gives $A(X_{\theta })\stackrel{d}{=}A(X_{{\theta }_0})$. Thus the standardized residual configuration provides a natural conditioning variable, while the equivariant fit contains the location and scale information.

Discarding zero observations and resolving ties is unnecessary with probability one under the stated independent continuous model. Rank the absolute observations from $1$ to $n$, let $R_i$ be the rank of $|X_i|$, and form the positive Wilcoxon signed-rank statisticReject the null hypothesis for large $W^{+}$. Under symmetry about zero, each sign is a fair independent binary variable, independent of all the absolute values. To see this independence, for every measurable positive set $B$, symmetry gives $\mathrm{Pr}(X\in B)=\mathrm{Pr}(X\in -B)=\frac{1}{2}\mathrm{Pr}(|X|\in B)$. Independence of observations then gives independence of the whole sign vector from the absolute-value vector. Conditional on those absolute values, relabeling by rank therefore givesThe null distribution is consequently independent of the unknown symmetric shape. Its probability generating function is $2^{-n}{\prod }_{r=1}^n(1+z^r)$, so the coefficient of $z^w$ gives the exact probability of $W^{+}=w$. Choose an upper critical value with null tail probability at most the desired level, or randomize at the boundary to obtain that level exactly. The observed upper-tail probability is an exact one-sided p-value. A positive translation raises each Walsh average, as established below, so this upper-tail rule has the correct direction against positive centers for every fixed symmetric shape.

The independent-sign representation provesThe asymptotic null distribution, stated without proof, isThis supplies the usual upper normal distribution critical value when exact calculation is inconvenient; for the discrete upper tail $\mathrm{Pr}(W^{+}\ge c)$, use $c-1/2$ in the continuity correction.

For exact equivalence with the proposed statistic, reorder the observations so $|X_{(1)}|<\cdots <|X_{(n)}|$. Partition the unordered pairs, including diagonals, by the member with larger absolute value. The block belonging to $X_{(r)}$ consists of the $r-1$ pairs with an earlier member and its diagonal pair. For an earlier member $X_{(j)}$, the strict inequality $|X_{(j)}|<|X_{(r)}|$ means the sign of $X_{(j)}+X_{(r)}$ is the sign of $X_{(r)}$; the diagonal sum has that same sign. Thus the entire block supplies exactly $r$ positive averages when $X_{(r)}>0$, and none otherwise. Summing the blocks provesThe two tests therefore have exactly the same statistic, critical values and p-values, not merely the same limiting distribution. The Walsh averages are dependent, so their positivity indicators must not instead be treated as an independent binomial sample. For a translation $X_i=Z_i+\theta$ of any null-symmetric sample, every Walsh average increases by $\theta$; their positive count is nondecreasing, which also verifies the direction of the one-sided test.

An ancillary statistic has a sampling distribution independent of the unknown parameter. It describes a realized aspect of the sample configuration without supplying a marginal likelihood for the parameter. Its observed value can nevertheless be crucial to the precision of inference from the rest of the data. In a transitive transformation model, invariance gives a common distribution under every parameter value, so maximal invariants provide natural ancillaries, such as the standardized configuration in a location-scale family.

Conditional inference uses the sampling distribution of an estimator or sufficient statistic conditional on the observed ancillary. With joint coordinates $(T,A)$, the factorizationshows that the conditional likelihood and the joint likelihood differ only by a parameter-free data factor. Conditional reference distributions, confidence intervals and p-values can still differ markedly from unconditional ones, because they adapt to the realized configuration. For a continuous ancillary one uses a regular conditional distribution or conditional density; dividing by the zero probability of a single ancillary value is not a valid definition.

An exact example makes this adaptation visible. Let independent observations be uniform on $(\theta -1/2,\theta +1/2)$, with $n\ge 2$. Put $U=X_{(1)}$, $V=X_{(n)}$, $R=V-U$ and $C=(U+V)/2$. Specifying the minimum and maximum contributes $n(n-1)$ choices of observations, while each remaining observation must lie between them. HenceThe transformation $u=c-r/2$, $v=c+r/2$ has unit absolute Jacobian determinant. Its support is $0<r<1$, $|c-\theta |<(1-r)/2$. Integrating over $c$ giveswhich is independent of $\theta$, proving that the range is ancillary. Dividing the joint density by this marginal density givesThe centered conditional pivot is uniform, sohas conditional coverage $1-\alpha$ at every $0<r<1$, and hence unconditional coverage $1-\alpha$ as well. A large realized range leaves little room for translating the interval and gives more precise inference. This is range-conditioned uniform location inference. The range and center are not independent: the conditional spread depends on the range, even though the range is ancillary.

Completeness can give independence in other models. Suppose $S$ is a complete sufficient statistic and $A$ is ancillary. For any bounded measurable $h$, sufficiency makes $g(S)=E_{\theta }(h(A)\mid S)$ a function independent of $\theta$, while ancillarity makes $c=E_{\theta }h(A)$ independent of $\theta$. Since $E_{\theta }(g(S)-c)=0$ for every $\theta$, completeness implies $g(S)=c$ almost surely. Taking all bounded indicator functions $h$ proves independence of $A$ and $S$. This is the content of Basu's theorem, with the proof displaying precisely where sufficiency and completeness are used. Without completeness the conclusion fails: in the uniform location example the minimum and maximum are sufficient, and contain the nondegenerate ancillary range.

There are practical qualifications. Ancillaries need not be unique, and some models supply only approximate ancillary coordinates; one must specify the conditioning surface and the accuracy intended. Conditioning on a statistic sufficient for a nuisance parameter is a different device: it may remove that nuisance even though the statistic is not ancillary for the full parameter. Conditioning indiscriminately can discard useful information. In higher-order likelihood inference, the ancillary coordinates specify how the fitted data are varied, entering both the p* density and the sample derivatives in the modified profile likelihood. Their role is therefore substantive, rather than merely a label for an extra statistic.

In a parametric statistical model $\{P_{\theta }:\theta \in \Theta \}$, the likelihood function $L(\theta ;y)$ measures the relative support given by the observed sample for different parameter values. Maximum likelihood estimation supplies a point estimate; the sampling distributions of suitable statistics calibrate significance tests and confidence intervals. A sufficient statistic retains all parameter-dependent information in the likelihood function, so one first seeks a reduction to such a statistic rather than discarding information arbitrarily.

The characteristic conditioning principle in Fisherian conditional inference is to calibrate inference conditional on the observed value of an ancillary statistic. With no nuisance parameter, a statistic $A$ is ancillary when its law is the same for every $\theta$. Its observed value can nevertheless describe the experimental configuration or precision. Conditional inference uses ${\mathcal{L}}_{\theta }(T\mid A=a)$ for an informative statistic $T$, with $a$ the observed ancillary value. Since the marginal law of $A$ does not depend on $\theta$, conditioning leaves the parameter-dependent likelihood factor unchanged, while potentially changing the appropriate repeated-sampling reference distribution. Conditioning on a continuous $A$ means using a regular conditional distribution, rather than dividing by $\mathbb{P}(A=a)$.

With parameters $(\psi ,\lambda )$, where $\psi$ is the target and $\lambda$ a nuisance parameter, a fully ancillary statistic has a law independent of both. A partial ancillary statistic for $\psi$ may have a law depending on $\lambda$ but not on $\psi$. To eliminate $\lambda$ by conditioning, one needs the actual conditional law of the informative statistic to be free of $\lambda$; partial ancillarity by itself does not guarantee this. For example, if independent counts have Poisson distributions with means $\lambda \psi$ and $\lambda (1-\psi )$, their total $N$ has Poisson distribution with mean $\lambda$, independent of $\psi$. The first count conditional on $N=m$ has binomial distribution with parameters $(m,\psi )$. Conditioning removes the nuisance intensity $\lambda$ exactly. More generally, nuisance-free pivotal quantities or conditional likelihoods can be used where available; there is no universal conditioning device that removes every nuisance parameter.

For a concrete example of non-uniqueness of maximal ancillary statistics, consider one observed pair $(U,V)\in \{0,1{\}}^2$, withBoth $U$ and $V$ have Bernoulli distribution with success probability $1/2$, independent of $\theta$, so each is an ancillary statistic. Their generated partitions are different and incomparable. Moreover, each is maximal: refining either two-point cell singles out a point whose probability is $\theta /2$ or $(1-\theta )/2$, which is parameter-dependent. Their joint statistic is not ancillary, since $P_{\theta }(U=V)=\theta$. Thus there is no common finer ancillary partition that reconciles the two choices. The conditional laws differ as labelled experiments: $P_{\theta }(V=U\mid U)=\theta$ and $P_{\theta }(U=V\mid V)=\theta$, but they condition on different observed information. This illustrates why “condition on a maximal ancillary” need not specify a unique procedure.

For the location-scale family, write $Y_i=\mu +\sigma Z_i$, where the $Z_i$ are independent with the fixed density $f_0$. Let $s_Z^2=n^{-1}{\sum }_i(Z_i-\bar{Z}{)}^2$. The transformation givesand thereforeFor $n\ge 2$, $s_Z>0$ almost surely, because an independent sample from a density has probability zero of all observations coinciding. The right side depends only on a sample from the fixed base law, so the entire vector $A$ has a law independent of $(\mu ,\sigma )$: $\boxed{A\text{ is ancillary}.}$ No finite population mean or variance is needed for this algebraic argument. With one observation the displayed standardization is undefined, so at least two observations are implicitly required.

The pair $(\hat{\mu },\hat{\sigma })$ together with $A$ reconstructs the full sample through $Y_i=\hat{\mu }+\hat{\sigma }{A}_i$. Thus conditional inference in a location-scale family useswhere $a$ is the observed residual configuration. Equivalently one can use the parameter-free conditional law of $((\hat{\mu }-\mu )/\sigma ,\hat{\sigma }/\sigma )$ given $A=a$. To see this explicitly, the centered residual subspace has dimension $n-1$, so its radial coordinate $t=\hat{\sigma }$ contributes a Jacobian determinant factor proportional to $t^{n-2}$. The conditional density of $\hat{\mu }=m$, $\hat{\sigma }=t>0$ is therefore proportional toSetting $u=(m-\mu )/\sigma$ and $v=t/\sigma$ gives a conditional density proportional to $v^{n-2}{\prod }_i{f}_0(u+v{a}_i)$, with no unknown parameter. The normalizing factor can depend on $a$, but not on $\mu$ or $\sigma$. These statements concern almost every attainable ancillary value.

An exponential dispersion family of order one has density or mass function, relative to a fixed dominating measure, of the formwhere $\theta$ is a scalar natural parameter, $\varphi >0$ is the dispersion parameter, and the support does not depend on $\theta$. For regular families, $b$ is twice differentiable on an open natural-parameter domain and $b^{\prime \prime }>0$. Known observation weights $w$ replace $\varphi$ by $\varphi /w$. “Order one” refers to the scalar canonical statistic $y$, rather than an arbitrary vector of canonical statistics.

Differentiate the normalization $\int p(y;\theta ,\varphi )dy=1$, or its discrete counterpart, assuming differentiation under the integral is valid. The first derivative gives $\mathbb{E}(Y-b^{\prime }(\theta ))=0$. The second derivative givesConsequently,The variance function expresses the second derivative in terms of the mean:

A generalized linear model specifies independent responses from an exponential dispersion family, a linear predictor ${\eta }_i=x_i^T\beta$, and a differentiable monotone link function relating it to the response mean by $g({\mu }_i)={\eta }_i$. The family, variance function, covariate design, link function and dispersion specification together define the model. The canonical link function identifies the linear predictor with the natural parameter:One advantage is that the likelihood equations become particularly simple. With known weights $w_i$ and common dispersion $\varphi$, the score function and negative Hessian matrix for $\beta$ areThe latter does not explicitly depend on the responses, so it equals the expected Fisher information matrix at the same parameter. With full-rank design and positive variances the likelihood is strictly concave in $\beta$, simplifying optimization. These statements concern the coefficient block at a fixed dispersion value.

Three explicit examples, taking unit weights, are as follows.

For a normal distribution with mean $\mu$ and variance $\varphi$,Thus $\theta =\mu$, $b(\theta )={\theta }^2/2$, $c(y,\varphi )=-y^2/(2\varphi )-\frac{1}{2}\log (2\pi \varphi )$, and $V(\mu )=1$. Its canonical link function is $\boxed{g(\mu )=\mu }$, the identity link.

For a Poisson distribution with mean $\mu >0$ and support $\{0,1,\dots \}$,Here $\varphi =1$, $\theta =\log \mu$, $b(\theta )=e^{\theta }$, and $c(y,1)=-\log (y!)$. Therefore $b^{\prime }(\theta )=b^{\prime \prime }(\theta )=e^{\theta }$, $V(\mu )=\mu$, and the Poisson canonical link is $\boxed{g(\mu )=\log \mu }$.

For a Bernoulli distribution with success probability $\mu \in (0,1)$,Taking $\varphi =1$, $\theta =\log (\mu /(1-\mu ))$, $b(\theta )=\log (1+e^{\theta })$, and $c(y,1)=0$ gives the required form. Its mean and variance function are $b^{\prime }(\theta )=e^{\theta }/(1+e^{\theta })=\mu$ and $V(\mu )=\mu (1-\mu )$. Its canonical link function is $\boxed{g(\mu )=\log (\mu /(1-\mu ))}$, the logit link. The fixed dispersion values in the Poisson and Bernoulli cases are part of those models; an arbitrary continuous dispersion is not being asserted for them.

To test $H_0:{\beta }_j=0$, first fit the unrestricted model and then the restricted model with that component fixed at zero, estimating all nuisance parameters, including dispersion when applicable, in each fit. If $\hat{\vartheta }$ and $\tilde{\vartheta }$ denote their full fitted parameter vectors, the likelihood-ratio test usesFor identifiable regular models, an interior null parameter and a fixed-dimensional full-rank design with increasing information, the Wilks theorem gives $D\stackrel{d}{\to }{\chi }_1^2$ under $H_0$. A level-$\alpha$ test rejects for $D$ greater than the $(1-\alpha )$-quantile of the chi-squared distribution with one degree of freedom.

Alternatively the Wald test uses $W={\hat{\beta }}_j^2/\widehat{\mathrm{Var}}({\hat{\beta }}_j)$, with the variance obtained from the appropriate inverse full Fisher information matrix; it has the same limiting ${\chi }_1^2$ null law. The score test fits only the restricted model. If $\lambda$ collects its nuisance parameters, its efficient information is $I_{jj\cdot \lambda }=I_{jj}-I_{j\lambda }{I}_{\lambda \lambda }^{-1}{I}_{\lambda j}$, and the statistic is $U_j(\tilde{\vartheta }{)}^2/I_{jj\cdot \lambda }(\tilde{\vartheta })$, again asymptotically ${\chi }_1^2$. Accounting for the nuisance block is necessary rather than using the unadjusted diagonal information.

For jackknife resampling, calculate the leave-one-out estimates ${\hat{\theta }}_{(-i)}$ by deleting observation $i$, and their average ${\overline{\theta }}_J=n^{-1}{\sum }_i{\hat{\theta }}_{(-i)}$. The jackknife variance estimator isThe factor $n-1$ compensates for the small differences between the overlapping samples. For sufficiently smooth statistics it estimates their sampling variance. The related estimated bias is $(n-1)({\overline{\theta }}_J-\hat{\theta })$, leading to the jackknife bias correction $n\hat{\theta }-(n-1){\overline{\theta }}_J$.

For the nonparametric bootstrap, replace the unknown distribution by its empirical distribution functionIndependently sample $n$ observations with replacement from the original observations, compute the estimator on that bootstrap sample, and repeat this for $B$ independently generated bootstrap samples. If the resulting values are ${\hat{\theta }}_1^{\ast },\dots ,{\hat{\theta }}_B^{\ast }$, estimate the conditional bootstrap variance byIts interpretation as an approximation to the original sampling variance relies on the bootstrap approximating the sampling law of the statistic; increasing $B$ only reduces simulation error and cannot repair a failure of that approximation.

A statistic $T$ is a sufficient statistic for $\theta$ if the conditional distribution of the data given $T$ does not depend on $\theta$. For a dominated family with density or mass function $f_{\theta }(x)$, the Fisher-Neyman factorization theorem states that this is equivalent towhere $h$ is nonnegative and independent of $\theta$, and $g_{\theta }$ is nonnegative and depends on the data only through $T$.

For a discrete sample space, suppose first that the factorization holds. If ${\mathbb{P}}_{\theta }(T=t)>0$, summing over the fibre gives ${\mathbb{P}}_{\theta }(T=t)=g_{\theta }(t){\sum }_{T(x)=t}h(x)$, soThis conditional distribution is independent of $\theta$, proving that $T$ is a sufficient statistic. Conversely, let $q_t(x)$ be the common conditional mass function guaranteed by the sufficient statistic property, for every fibre attainable under at least one parameter. Set $h(x)=q_{T(x)}(x)$ and $g_{\theta }(t)={\mathbb{P}}_{\theta }(T=t)$. Then ${\mathbb{P}}_{\theta }(X=x)=g_{\theta }(T(x))h(x)$. Fibres never attained under any parameter can be assigned any conditional distribution, since their multiplying probability is always zero. This proves both directions of the discrete criterion, including possible parameter-dependent supports.