= Statistical data compression by a sufficient statistic
A <sufficient statistic> allows the observed <likelihood function> to be stored through a shorter record, up to a parameter-independent factor. For the <Bernoulli sample sum as a sufficient statistic>, write
$$
L_x(p)=p^t(1-p)^{n-t},\qquad L_t(p)=\binom nt p^t(1-p)^{n-t}=\binom nt L_x(p).
$$
The second <likelihood function> belongs to the observed count; the first belongs to the observed sequence. Their absolute values differ by the <binomial coefficient>, but their <likelihood ratios> between any two interior parameter values agree. A <maximum-likelihood estimator> can therefore use just $(n,t)$.
\Image[/bernoulli-sufficient-statistic-likelihood-compression.png]
{title=Sequence to count: sufficient compression preserves likelihood comparisons}
{description=For $n=4,t=2$, three example sequences share $L_x(p)=p^2(1-p)^2$. Recording $(n,t)$ produces the count <likelihood> $L_t(p)=6L_x(p)$. Normalizing each curve by its maximum makes the two curves coincide. A reverse random draw from the six arrangements restores the full-data <probability distribution>, using no unknown $p$.}
This compression discards which arrangement actually occurred. To reproduce the original experiment's <probability distribution>, first draw $T$ from its <binomial distribution> and then draw uniformly from the arrangements with that sum. The second step is independent of $p$, so it supplies no additional evidence about $p$. It does not recover the particular discarded sequence. If the scientific question changes to time trends or <statistical dependence> between observations, the common-$p$ <independent and identically distributed> <Bernoulli> model may no longer apply, and the ordering can become informative.
Back to article page
