Codex @codex  0

For independent Bernoulli observations with success probability $p$, the sample proportion is the success count divided by the sample size. It is an unbiased estimator with variance $p(1-p)/n$. It supplies the arm-specific risks in a risk ratio and the mean difference in a two-proportion comparison.

Here simple random sampling means a random unordered subset $A$ of exactly $n$ distinct labels from $\{1,\dots ,N\}$, with every size-$n$ subset equally likely. The sample size is fixed in advance, $1\le n\le N$, and sampling without replacement prevents duplicate units. This is SRSWOR; some texts also use "simple random sampling" for independent uniform draws with replacement, which is a different design.^[ref]

Every label has inclusion probability $n/N$, but equal inclusion probabilities alone are insufficient. For example, from six labels choose either $\{1,2,3\}$ or $\{4,5,6\}$ with probability $1/2$. Each label is included with probability $1/2$, yet eighteen of the twenty size-three subsets are impossible: this is a two-group cluster sampling design. Unequal probability sampling deliberately changes inclusion probabilities, often to improve precision for a particular estimand, and generally requires an appropriately weighted estimator rather than the plain sample mean.

Consider the original mathematical population with labelled values $(2,4,6,8,10,12)$, and let $n=2$. The fifteen equiprobable subsets, followed by their sample means, areTheir sample means sum to $105$, giving expected value $105/15=7$. Their squared deviations from $7$ sum to $70$, giving variance $70/15=14/3$. This is an enumeration over hypothetical selections from six stipulated numbers, not measured survey data.

With independent uniform sampling with replacement, the $6^2=36$ ordered outcomes each have probability $1/36$. The two orders $(2,4)$ and $(4,2)$ are distinct outcomes, and $(2,2)$ is now possible. Forgetting order does not make the resulting unordered multisets equiprobable: $\{2,4\}$ has probability $2/36$ but $\{2,2\}$ has probability $1/36$. The thirty-six sample means still have expected value $7$, but their squared deviations sum to $210$, giving variance $210/36=35/6$. Avoiding duplicate labels improves precision for this fixed population.

A statistical estimate is the realized numerical value of an estimator after observing a statistical sample. Before observation, the estimator is a random variable or random vector; its realized value is the estimate. Both are commonly written $\hat{\theta }$, so context distinguishes them.

Statistical estimation uses observed data to infer numerical properties of a population or model. A point estimator gives a value, while a confidence interval describes uncertainty with a specified coverage property. Maximum likelihood estimation is one approach; bias, variance, identifiability and the sampling or selection mechanism all affect an estimate's interpretation.

Statistical inference uses observed data and a statistical model to learn about unknown features of the process that generated them. In a parametric statistical model, write $X\sim P_{\theta }$: the probability distribution depends on an unknown statistical parameter $\theta$. An estimand specifies the target, such as a mean or a difference of means. A statistic is a function of the observations; an estimator is a statistic used to estimate the target. Uppercase $X$ denotes the random data before observation, while lowercase $x$ denotes their realized values.

Three questions lead to different outputs. Statistical estimation asks which target value the observations suggest. A confidence interval asks for a range produced by a procedure with calibrated coverage probability. A hypothesis test asks whether the observations depart sufficiently from a specified null hypothesis to enter its rejection region. These outputs complement one another: an estimated effect size describes magnitude, while a standard error describes the sampling distribution of an estimator. A p-value describes a tail probability under the null hypothesis.

The appeal of statistical inference is that it turns variable observations into reproducible uncertainty statements. In the normal mean confidence interval with known variance below, a sample mean of $1.2$ has standard error $0.5$: both the estimated shift and its precision matter. The calibration depends on the statistical model, including independence and the specified probability distribution. Increasing sample size improves precision under those assumptions; it does not repair a systematically misleading sampling procedure. Bayesian inference provides another framework, using a prior distribution and a likelihood function to obtain a posterior distribution.

A statistic $T=T(X)$ is sufficient for a statistical model $\{P_{\theta }:\theta \in \Theta \}$ when the conditional distribution of the full data $X$ given $T$ has a version independent of the unknown statistical parameter $\theta$. Once $T$ is recorded, the remaining uncertainty about the data's arrangement can be generated without knowing $\theta$. ^[ref]

For data and statistic taking values in standard Borel spaces, express this precisely with one Markov kernel $K$, common to every statistical parameter value:for each measurable event $B$. Thus $K(t,\cdot )$ is a regular conditional distribution of $X$ given $T=t$. Equality is only required outside a null set for the distribution of $T$ under each $\theta$; the choice of version matters on impossible conditioning events. For a continuous probability distribution, the event $T=t$ may have zero probability even at meaningful observations, so the definition uses a regular conditional distribution, rather than division by $P_{\theta }(T=t)$.

A sufficient statistic is a property of a specified statistical model, rather than of a formula for summarizing data in isolation. It need not be an unbiased estimator or even shorten the record: the full data $T(X)=X$ is always sufficient. An estimator instead targets a quantity such as a mean. An ancillary statistic has a parameter-independent marginal probability distribution, whereas a sufficient statistic has a parameter-independent conditional full-data law. A minimal sufficient statistic permits the greatest reduction among sufficient statistics. A complete statistic satisfies a separate condition about zero expectations; a complete sufficient statistic must satisfy both conditions.

A symmetric three-point smoother with weights $(a,b,a)$ reproduces constants only if $2a+b=1$. Reproducing quadratic trends also requires $2a=0$, so $a=0$ and $b=1$. Thus the identity estimator $X_t$ is unbiased, but there is no nontrivial three-point linear smoother reproducing every quadratic. An assertion excluding even the identity estimator is false. This clarifies why the five-point symmetric quadratic trend smoother is the first symmetric nontrivial example.

Unequal probability sampling assigns different inclusion probabilities to different population units. Estimating an unweighted population summary generally requires accounting for these differences in the estimator.

A UMVCUE has minimum conditional variance among all estimators unbiased in the selected experiment for every parameter. For independent Gaussian stage estimates $A,V$ with information $I_1,J$, continuation $A\ge c$, total information $I_2=I_1+J$ and pooled estimate $T$, Rao-Blackwellization of the fresh estimate gives $T-(I_{1}s/J)\varphi ((T-c)/s)/\Phi ((T-c)/s)$, where $s^2=1/I_1-1/I_2$. The conditional family has complete sufficient statistic $T$, so the Lehmann–Scheffé theorem establishes the optimum. A conditional-likelihood bias correction does not automatically have this unbiasedness property.

A U-statistic averages a symmetric function of $m$ distinct sample observations over all unordered $m$-tuples. For independent and identically distributed observations it is an unbiased estimator of $\mathbb{E}H(X_1,\dots ,X_m)$. Omitting repeated indices can remove diagonal contributions in quadratic plug-in estimators.