Sufficient statistic (${\mathcal{L}}_{\theta }(X\mid T)=K(T,\cdot )$)

A statistic $T=T(X)$ is sufficient for a statistical model $\{P_{\theta }:\theta \in \Theta \}$ when the conditional distribution of the full data $X$ given $T$ has a version independent of the unknown statistical parameter $\theta$. Once $T$ is recorded, the remaining uncertainty about the data's arrangement can be generated without knowing $\theta$. ^[ref]

For data and statistic taking values in standard Borel spaces, express this precisely with one Markov kernel $K$, common to every statistical parameter value:for each measurable event $B$. Thus $K(t,\cdot )$ is a regular conditional distribution of $X$ given $T=t$. Equality is only required outside a null set for the distribution of $T$ under each $\theta$; the choice of version matters on impossible conditioning events. For a continuous probability distribution, the event $T=t$ may have zero probability even at meaningful observations, so the definition uses a regular conditional distribution, rather than division by $P_{\theta }(T=t)$.

A sufficient statistic is a property of a specified statistical model, rather than of a formula for summarizing data in isolation. It need not be an unbiased estimator or even shorten the record: the full data $T(X)=X$ is always sufficient. An estimator instead targets a quantity such as a mean. An ancillary statistic has a parameter-independent marginal probability distribution, whereas a sufficient statistic has a parameter-independent conditional full-data law. A minimal sufficient statistic permits the greatest reduction among sufficient statistics. A complete statistic satisfies a separate condition about zero expectations; a complete sufficient statistic must satisfy both conditions.

Let $n$ be fixed and known, and let $X_1,\dots ,X_n$ be independent and identically distributed Bernoulli random variables with common success probability $p$. Their joint probability mass function, on the binary sample space $\{0,1{\}}^n$, isThere are $\left(\genfrac{}{}{0ex}{}{n}{t}\right)$ arrangements with $t$ successes, so $T$ has the binomial distribution. For $0<p<1$ and $t\in \{0,\dots ,n\}$, its exact conditional full-data probability mass function isThis conditional distribution is uniform over arrangements, with no dependence on $p$: the sum is a sufficient statistic. ^[ref]

For $n=4$, the group sizes are $1,4,6,4,1$ and the within-group conditional probabilities are respectively $1,1/4,1/6,1/4,1$. For example, $T=2$ leaves exactly $0011,0101,0110,1001,1010,1100$, each with conditional probability $1/6$. At $p=1/4$, each such sequence has unconditional probability $9/256$ and the group has probability $54/256$. At $p=1/2$, these values are $1/16$ and $6/16$; at $p=3/4$ they return to $9/256$ and $54/256$. The conditional weights stay $1/6$ in all three cases.

At the boundary $p=0$, only $T=0$ has positive probability; at $p=1$, only $T=n$ does. Dividing by $P_p(T=t)$ on the other groups would divide by zero. Nevertheless, the same uniform-on-arrangements Markov kernel is a valid common version for the entire family $0\le p\le 1$: it gives the correct point mass on the one observed group at each boundary and assigns permissible versions on its unobserved groups. Define the joint probability mass function there by the original product of the one-trial Bernoulli probability mass functions, or use the limiting convention $0^0=1$ in the compact formula.

The sample mean $T/n$ is an unbiased estimator of $p$ and retains the sufficient statistic because $n$ is known. By contrast, $X_1$ is also an unbiased estimator of $p$ but, for $n\ge 2$, is not sufficient: the conditional success probability of the independent $X_2$ given $X_1$ is still $p$. Thus unbiasedness does not imply sufficiency.

Sufficiency also does not imply completeness of a statistic. For $n\ge 2$, the full-data sufficient statistic $X$ admits the nonzero function $g(X)=X_1-X_2$ with expectation zero for every $p$. Nor does minimality imply completeness of a statistic: for $n=2$ and the restricted statistical model $p\in \{1/4,3/4\}$, $T$ is a minimal sufficient statistic since the likelihood ratio $P_{3/4}(X=x)/P_{1/4}(X=x)=3^{2t-2}$ distinguishes its three values. Yet $g(0)=3,g(1)=-5,g(2)=3$ has expectation $(27-30+3)/16=0$ at both parameter values and is never zero. Changing the parameter family can change completeness of a statistic even while preserving sufficiency.

A sufficient statistic allows the observed likelihood function to be stored through a shorter record, up to a parameter-independent factor. For the Bernoulli sample sum as a sufficient statistic, writeThe second likelihood function belongs to the observed count; the first belongs to the observed sequence. Their absolute values differ by the binomial coefficient, but their likelihood ratios between any two interior parameter values agree. A maximum-likelihood estimator can therefore use just $(n,t)$.

This compression discards which arrangement actually occurred. To reproduce the original experiment's probability distribution, first draw $T$ from its binomial distribution and then draw uniformly from the arrangements with that sum. The second step is independent of $p$, so it supplies no additional evidence about $p$. It does not recover the particular discarded sequence. If the scientific question changes to time trends or statistical dependence between observations, the common-$p$ independent and identically distributed Bernoulli model may no longer apply, and the ordering can become informative.

A statistic is ancillary for a statistical model if its probability distribution is the same for every parameter value. Its marginal law carries no information about the parameter, but its observed value can identify the precision or configuration of the experiment. Fisherian conditional inference uses the conditional distribution of an informative statistic given the observed ancillary value. Ancillarity does not mean that the ancillary is independent of every informative statistic.

For independent observations uniform on $(\theta -1/2,\theta +1/2)$, let $U$ and $V$ be the sample minimum and maximum, $R=V-U$, and $C=(U+V)/2$. For $n\ge 2$ their joint density in $(c,r)$ is $n(n-1)r^{n-2}$ on $0<r<1$, $|c-\theta |<(1-r)/2$. Integrating over $c$ gives $f_R(r)=n(n-1)r^{n-2}(1-r)$, independent of $\theta$, so $R$ is an ancillary statistic. Moreover $C\mid R=r$ is uniform on $(\theta -(1-r)/2,\theta +(1-r)/2)$. Consequentlyis an exact conditional $(1-\alpha )$ confidence interval for $\theta$. The realized range determines the appropriate precision, although the range is not itself evidence about the value of $\theta$.

A maximal ancillary statistic has no strictly finer ancillary partition of the sample space, but different maximal ancillary partitions can exist. Let $(U,V)$ take values in $\{0,1{\}}^2$, with probabilities $\theta /2$ on each equal pair and $(1-\theta )/2$ on each unequal pair, for $0<\theta <1$. Both $U$ and $V$ have the Bernoulli distribution with success probability $1/2$ for every $\theta$. Their joint law depends on $\theta$, so combining them is not ancillary. Each partition is maximal: splitting one of its two-point cells singles out a point whose probability depends on $\theta$. There is no common ancillary refinement, and conditioning on one ancillary choice need not specify a unique conditioning principle for all models.

For a parameter of interest $\psi$ and nuisance parameter $\lambda$, a partial ancillary for $\psi$ has a marginal law independent of $\psi$, possibly depending on $\lambda$. This differs from a fully ancillary statistic, whose law is independent of both. For independent Poisson counts with means $\lambda \psi$ and $\lambda (1-\psi )$, their total is Poisson with mean $\lambda$, while the first count conditional on the total is binomial with success probability $\psi$. Conditioning on the total eliminates $\lambda$. A partial ancillary does not automatically eliminate a nuisance parameter; the conditional law must actually be free of that parameter.

For a sufficient statistic $(C,S)$ and parameter $(\psi ,\lambda )$, a useful strong reduction requires the law of $S$ to be free of $\psi$ and the conditional distribution of $C$ given $S$ to be free of the nuisance parameter $\lambda$. In the discrete case it giveswhere ${\varphi }_0$ sums to one on each $(C,S)$ fiber, ${\varphi }_S$ sums to one over $s$, and ${\varphi }_C$ sums to one over $c$ separately for every fixed $s$. Ordinary ancillarity, or just the law of $S$ being free of $\psi$, does not by itself guarantee that conditioning removes $\lambda$.

A sufficient statistic is complete when every integrable function of it with expectation zero for every parameter is almost surely zero for every parameter. Full regular one-parameter exponential families with natural parameter ranging over an open interval provide a standard completeness result. Completeness makes an unbiased function of a sufficient statistic unique, giving the Lehmann–Scheffé theorem.

A complete sufficient statistic is independent of every ancillary statistic. To prove this, let $S$ be complete and sufficient, let $A$ be ancillary, and take any bounded measurable function $h$. By sufficiency, $g(S)=E_{\theta }[h(A)\mid S]$ has a parameter-independent version; by ancillarity, $c=E_{\theta }h(A)$ is independent of $\theta$. Thus $E_{\theta }[g(S)-c]=0$ for every $\theta$. Completeness gives $g(S)=c$ almost surely. Choosing $h$ to be the indicator of each measurable event in the ancillary space proves independence. Completeness is essential: a sufficient statistic may itself contain a nonconstant ancillary, as in the minimum and maximum of a uniform location sample.

For independent observations $X_i\sim N(\mu ,{\sigma }^2)$ with known variance, ${\sum }_i{X}_i$ is minimal sufficient for $\mu$. Factorization proves sufficiency, while the likelihood ratio for two samples is independent of $\mu$ exactly when their sums agree.

A sufficient statistic is minimal sufficient when it is a function of every other sufficient statistic, up to null sets. Its level sets form the coarsest sufficient partition of the sample space.

For independent samples uniform on $[-\theta ,\theta ]$ with $\theta >0$, the absolute maximum $M$ is a minimal sufficient statistic, since the likelihood is $(2\theta {)}^{-n}{1}_{\{\theta \ge M\}}$. The pair of sample minimum and maximum is a sufficient statistic but is not minimal: simultaneous sign reversal preserves $M$ and generally changes that pair.

For a dominated family with positive densities $p_{\theta }$, a statistic $T$ is minimal sufficient if

Assume the full-data sample space and the range of the statistic $T$ are standard Borel spaces, $T$ is a measurable function, and the statistical model $\{P_{\theta }\}$ is dominated by one common sigma-finite measure $\nu$. Write $f_{\theta }=d{P}_{\theta }/d\nu$ for its densities. Then $T$ is a sufficient statistic if and only if there exist nonnegative measurable functions $g_{\theta }$ and $h$, with $h$ independent of $\theta$, such thatThe statistical parameter can enter $g_{\theta }$ only through $\theta$ and the observed $T(x)$. ^[ref]

For discrete data, take counting measure; for ordinary continuous data, take Lebesgue measure. Independence, identical sampling laws, smooth dependence on $\theta$, and a fixed positive support of a probability distribution are not requirements of this theorem. Parameter-dependent support must nevertheless be included in $f_{\theta }$, usually through an indicator function, and must fit the displayed factorization. The sufficient statistic for a symmetric uniform sample illustrates exactly this: its indicator function depends on the unknown scale through the observed absolute maximum. The null sets in the theorem concern versions of densities; the factors are not generally themselves normalized probability distributions.

For a discrete sample space, the cancellation behind the theorem is particularly transparent. On a group with positive probability, factorization givesConversely, multiply a common conditional probability mass function by $P_{\theta }(T=T(x))$ to obtain the required factorization. For continuous data, a sum over a group is not a density integral over the original Lebesgue measure: the group may have measure zero. Use the regular conditional distribution formulation and the Fisher-Neyman factorization theorem instead of that discrete calculation.

For the normal sample sum with known variance, assume independent and identically distributed $X_i\sim N(\mu ,{\sigma }^2)$ with fixed known variance ${\sigma }^2>0$. The joint probability density factors asThe Fisher-Neyman factorization theorem proves that the sum, and hence the sample mean, is a sufficient statistic for $\mu$. Its remaining deviations $X_i-\overline{X}$ have a joint Gaussian law independent of $\mu$ and are independent of $\overline{X}$. If both $\mu$ and ${\sigma }^2$ are unknown, the sample mean alone loses information about variance; the pair $({\sum }_i{X}_i,{\sum }_i{X}_i^2)$ is sufficient for the full statistical parameter.

A discrete counterexample shows why a sample mean is not automatically sufficient for arbitrary statistical models. Let $X_1,X_2$ be independent and identically distributed on $\{0,1,2\}$ withEach observation has mean one. The arrangements $(0,2),(1,1),(2,0)$ all give sample mean one, yetThis conditional probability is $18/19$ at $\theta =1/4$ and $2/11$ at $\theta =3/4$. The conditional distribution therefore still depends on the unknown statistical parameter: this sample mean is not a sufficient statistic. The useful reduction depends on the family of full-data laws, even when its mean stays fixed.