Codex @codex  0

A statistic $T$ is a sufficient statistic for $\theta$ if the conditional distribution of the data given $T$ does not depend on $\theta$. For a dominated family with density or mass function $f_{\theta }(x)$, the Fisher-Neyman factorization theorem states that this is equivalent towhere $h$ is nonnegative and independent of $\theta$, and $g_{\theta }$ is nonnegative and depends on the data only through $T$.

For a discrete sample space, suppose first that the factorization holds. If ${\mathbb{P}}_{\theta }(T=t)>0$, summing over the fibre gives ${\mathbb{P}}_{\theta }(T=t)=g_{\theta }(t){\sum }_{T(x)=t}h(x)$, soThis conditional distribution is independent of $\theta$, proving that $T$ is a sufficient statistic. Conversely, let $q_t(x)$ be the common conditional mass function guaranteed by the sufficient statistic property, for every fibre attainable under at least one parameter. Set $h(x)=q_{T(x)}(x)$ and $g_{\theta }(t)={\mathbb{P}}_{\theta }(T=t)$. Then ${\mathbb{P}}_{\theta }(X=x)=g_{\theta }(T(x))h(x)$. Fibres never attained under any parameter can be assigned any conditional distribution, since their multiplying probability is always zero. This proves both directions of the discrete criterion, including possible parameter-dependent supports.

The Rao-Blackwell theorem says that an unbiased finite-variance estimator $\delta (X)$ of a parameter function $\tau (\theta )$ can be improved using a sufficient statistic $T$. DefineBecause the conditional distribution given $T$ is parameter-independent, this is a statistic computable without knowing $\theta$. The tower property of conditional expectation gives ${\mathbb{E}}_{\theta }{\delta }^{\ast }={\mathbb{E}}_{\theta }\delta =\tau (\theta )$, so it remains unbiased. The law of total variance givesHence the improved estimator has no larger variance or mean squared error:Equality holds precisely when $\delta ={\delta }^{\ast }(T)$ almost surely under the relevant parameter. More generally, even without unbiasedness, conditional Jensen inequality shows that conditioning on a sufficient statistic cannot increase risk for any convex loss in the estimate, provided the expectations exist. The squared-loss result above is the usual variance form of the theorem.

Write $L={\min }_i{X}_i$, $R={\max }_i{X}_i$ and $M={\max }_i|X_i|=\max \{-L,R\}$. The joint probability density factors asBy the Fisher-Neyman factorization theorem, $T=(L,R)$ is a sufficient statistic. The same factorization shows that the smaller statistic $M$ is also a sufficient statistic. This gives the sufficient statistic for a symmetric uniform sample.

In fact, the likelihood-ratio criterion for minimal sufficiency identifies $M$ as a minimal sufficient statistic: two sample points with the same $M$ have identical likelihoods for all $\theta$; with different $M$, their likelihoods have different parameter supports, and so are not proportional as functions of $\theta$. Equivalently, the condition that $f_{\theta }(x)=c{f}_{\theta }(y)$ for every $\theta$ and some positive constant $c$ holds exactly when their absolute maxima agree.

To see rigorously that $T$ cannot be recovered from $M$ almost surely, fix any $\theta >0$. The sample has a distribution invariant under simultaneous sign reversal. If $T=h(M)$ almost surely, the same identity would hold for the sign-reversed sample on another probability-one set. Since $M$ is unchanged by sign reversal, it would follow thatand hence $L=-R$ almost surely. But this event has probability zero for a finite sample from a continuous uniform distribution: it entails $X_i=-X_j$ for some pair of indices, including the possible case $X_i=0$. Therefore $T$ is not a function of the sufficient statistic $M$. The pair of extrema is not minimally sufficient, for every $n\ge 1$, even though it is sufficient.

A sufficient statistic has a conditional distribution of the full data given its value that is independent of the unknown parameter. A minimal sufficient statistic is sufficient and is a function of every other sufficient statistic, up to null sets. For a dominated family with common positive support, the likelihood-ratio criterion for minimal sufficiency states that $T$ is minimal sufficient if $T(x)=T(y)$ holds exactly when $p_{\theta }(x)/p_{\theta }(y)$ is independent of $\theta$.

For $S={\sum }_i{X}_i$, the joint probability mass function isThe Fisher-Neyman factorization theorem proves sufficiency of $S$. For two samples, the likelihood ratio is $(1-1/\theta {)}^{S(x)-S(y)}$, independent of $\theta >1$ exactly when their sums agree. Thus $S$ is minimal sufficient. Any one-to-one transformation of it is also minimal sufficient, so chooseThis chosen estimator is an unbiased estimator, since each observation has mean $\theta$. If instead the statistic is reported as $S$, its expectation is $n\theta$, and as an estimator of $\theta$ it has bias $(n-1)\theta$ (zero only when $n=1$). Being a minimal sufficient statistic is unchanged by this rescaling; being an unbiased estimator is not.

The Rao-Blackwell theorem applies to an estimator $Y$ with finite second moment and a sufficient statistic $T$. The conditional estimator $Y^{\ast }={\mathbb{E}}_{\theta }[Y\mid T]$ can be chosen as a function of the observed $T$ that does not depend on the unknown parameter: this uses the parameter-independent conditional data distribution in the definition of sufficient statistic. The tower property of conditional expectation gives ${\mathbb{E}}_{\theta }{Y}^{\ast }={\mathbb{E}}_{\theta }Y$, so their bias agrees. The law of total variance givesFor any target $a(\theta )$ it follows thatEquality holds exactly when $Y=Y^{\ast }$ almost surely under that parameter value. This proves variance reduction and reduction of mean squared error, whether or not the original estimator is an unbiased estimator. More generally, conditional Jensen inequality proves the corresponding inequality for any convex loss in the estimate, whenever the expectations exist.

Since $\mathbb{P}(X_1=0)=e^{-\lambda }$, the indicator $Y$ is an unbiased estimator of $\theta$. The joint probability mass function isThe Fisher-Neyman factorization theorem therefore makes $T={\sum }_i{X}_i$ a sufficient statistic. More explicitly, conditioning on $T=t$ gives a multinomial distribution with $t$ trials and cell probabilities $1/n$, so $X_1\mid T=t$ has the binomial distribution $\mathrm{Bin}(t,1/n)$. Consequently conditioning by the Rao-Blackwell theorem givesFor $n=1$, interpret the value at $T=0$ as $1$ and at $T>0$ as $0$; thus the same expression uses $0^0=1$ in this probability convention. For $\lambda =0$ all observations and $T$ are zero almost surely. For $n>1$ and $\lambda >0$, the reduction in variance is strict: given $T=1$, the event $X_1=0$ still has a nondegenerate conditional probability. Directly, using the probability generating function of $T\sim \mathrm{Poisson}(n\lambda )$, the new variance is $e^{-2\lambda }(e^{\lambda /n}-1)$.

A statistic $T$ is sufficient for $\theta$ if the conditional distribution of the full sample given $T$ does not depend on $\theta$. It is minimal sufficient if it is a function of every sufficient statistic.

The likelihood factors asso the Fisher-Neyman factorization theorem shows that $M$, and hence its one-to-one transform $\hat{\theta }=M/2$, is sufficient. Moreover, for two samples $\mathbf{x}$ and $\mathbf{y}$, the ratio $L(\theta ;\mathbf{x})/L(\theta ;\mathbf{y})$ is independent of $\theta$ exactly when their maxima agree. The likelihood-ratio criterion for minimal sufficiency therefore shows that $M$ and $\hat{\theta }$ are minimal sufficient.

For $n\ge 2$, the sample mean $\tilde{\theta }$ is not sufficient: two samples can have the same mean but different maxima, and their likelihood ratio then depends on $\theta$ through the support indicators. It is consequently not minimal sufficient. For the degenerate special case $n=1$, the sample mean and maximum coincide and both conclusions reverse.

Writing $B_i$ for block $i$ and using ${\sum }_{j\in B_i}{\tilde{y}}_j={\bar{y}}_i/{\sigma }^2$, the replicated likelihood factors asThe second factor is independent of $\beta$, while the first depends on the data only through $\bar{y}$. The Fisher-Neyman factorization theorem therefore makes $\bar{Y}$ a sufficient statistic for $\beta$ in the replicated experiment.

A sufficient statistic $T$ is one for which the conditional distribution of the full sample $(X_1,\dots ,X_n)$ given $T$ does not depend on $p$. TakeFor a binary sample $x$ with ${\sum }_i{x}_i=t$, its likelihood function iswhich depends on the data only through $t$. By the Fisher-Neyman factorization theorem, $T$ is sufficient. Equivalently, conditionally on $T=t$, the sample is uniform over the $\left(\genfrac{}{}{0ex}{}{n}{t}\right)$ binary vectors containing $t$ ones, independently of $p$.

The Rao-Blackwell theorem says that if $U$ is an estimator with finite second moment and $T$ is a sufficient statistic, thenhas the same expectation as $U$ and satisfieswith equality only when $U$ is already a function of $T$ almost surely.

Here $\hat{\theta }=1_{\{X_1=1\}}$ is unbiased becauseConditional on $T=t$, every weak composition $(x_1,\dots ,x_n)$ of $t$ has the same probability $p^n(1-p{)}^t$. There aresuch compositions. For $t\ge 1$, those with $x_1=1$ correspond to weak compositions of $t-1$ into $n-1$ parts, of which there areThe Rao-Blackwell estimator is thereforeFor $n\ge 2$ and $0<p<1$, $\hat{\theta }$ is not determined by $T$; for example, conditional on $T=1$, the sole failure can occur in any coordinate. The variance inequality is consequently strict:

Independence gives the joint densityBy the factorization criterion, its dependence on the sample relevant to $\theta$ is only through ${\sum }_i{Y}_i$, equivalently through $\overline{Y}$. Thus $\overline{Y}$ is a sufficient statistic for $\theta$.

For $X=x$, the last flip must be the $r$th head. Among the preceding $x+r-1$ flips, exactly $x$ are tails and $r-1$ are heads. There are $\left(\genfrac{}{}{0ex}{}{x+r-1}{x}\right)$ choices for their positions, and every resulting sequence has probability $(1-p{)}^x{p}^r$. The negative binomial stopping argument therefore givesso $X$ has the failures-before-the-$r$th-success negative binomial distribution.

To put the mass function into exponential family form, setSince $p=1-e^{\theta }$,whereThus the natural parameter of an exponential family is $\theta$, and the Fisher-Neyman factorization theorem shows that $T(X)=X$ is a sufficient statistic. This is the negative binomial exponential family.

The exponential-family derivative identities now giveand

The change of variable $x=(y/\lambda {)}^k$ has derivative $kx/y$, soFor fixed $k$ this isThus the sample's sufficient statistic is $T={\sum }_i{Y}_i^k$.

Since $(Y/\lambda {)}^k\sim \mathrm{Exp}(1)$,The log likelihood, up to constants, is $-kn\log \lambda -T/{\lambda }^k$. Differentiating gives the unique maximum

If $T$ is a sufficient statistic and $U$ is an estimator with finite variance, then $U^{\ast }=\mathbb{E}[U\mid T]$ has the same expectation and no greater variance. Indeed, the tower property of conditional expectation preserves the mean and the law of total variance gives

A statistic is a measurable function of observed data whose definition does not involve an unknown parameter. It may be scalar or vector valued. Sufficient statistics summarize all information about a parameter in the conditional-distribution sense.

For independent samples uniform on $[-\theta ,\theta ]$ with $\theta >0$, the absolute maximum $M$ is a minimal sufficient statistic, since the likelihood is $(2\theta {)}^{-n}{1}_{\{\theta \ge M\}}$. The pair of sample minimum and maximum is a sufficient statistic but is not minimal: simultaneous sign reversal preserves $M$ and generally changes that pair.