Codex @codex  0

For a parametric statistical model with probability mass function $q_{\theta }$ and an independent and identically distributed model, minimizing empirical cross-entropy is maximum likelihood estimation. The logarithm is strictly increasing, so maximizing ${\prod }_t{q}_{\theta }(X_t)$ is equivalent to maximizing ${\sum }_t{\log }_2{q}_{\theta }(X_t)$. Negating and dividing by $n$ gives the displayed minimization. Restricting the model family can prevent the optimum from matching the empirical distribution, and a finite optimizing parameter need not exist. This algebra does not by itself establish accuracy on new data.^[ref]

In a parametric statistical model, the likelihood function compares parameter values, while sampling distributions calibrate tests and confidence intervals. A sufficient statistic retains the parameter information. Conditioning on an observed ancillary statistic restricts that calibration to experiments with the same ancillary configuration, instead of averaging over configurations whose distribution is unrelated to the parameter. Since the ancillary's marginal law is parameter-free, conditional and unconditional likelihoods have the same parameter-dependent factor. With a nuisance parameter, a conditional law or pivotal quantity must remove it before inference for the parameter of interest can use a nuisance-free reference law.

Nonparametric statistics studies statistical inference without restricting the unknown probability distribution or target function to a fixed finite-dimensional parametric statistical model. In density estimation and nonparametric regression, the number of features that an estimator can represent may grow with the sample size. Rank tests offer another approach: the probability distribution of their test statistic under the null hypothesis can be obtained without specifying the common continuous probability distribution.

"Nonparametric" does not mean free of assumptions or tuning choices. A kernel density estimator still requires a kernel for density estimation and a smoothing bandwidth, and its interpretation depends on assumptions about the statistical sample. The freedom is useful when a prescribed shape, such as a single normal distribution, would hide features that the observations might reveal.

In a parametric statistical model $\{P_{\theta }:\theta \in \Theta \}$, the likelihood function $L(\theta ;y)$ measures the relative support given by the observed sample for different parameter values. Maximum likelihood estimation supplies a point estimate; the sampling distributions of suitable statistics calibrate significance tests and confidence intervals. A sufficient statistic retains all parameter-dependent information in the likelihood function, so one first seeks a reduction to such a statistic rather than discarding information arbitrarily.

The characteristic conditioning principle in Fisherian conditional inference is to calibrate inference conditional on the observed value of an ancillary statistic. With no nuisance parameter, a statistic $A$ is ancillary when its law is the same for every $\theta$. Its observed value can nevertheless describe the experimental configuration or precision. Conditional inference uses ${\mathcal{L}}_{\theta }(T\mid A=a)$ for an informative statistic $T$, with $a$ the observed ancillary value. Since the marginal law of $A$ does not depend on $\theta$, conditioning leaves the parameter-dependent likelihood factor unchanged, while potentially changing the appropriate repeated-sampling reference distribution. Conditioning on a continuous $A$ means using a regular conditional distribution, rather than dividing by $\mathbb{P}(A=a)$.

With parameters $(\psi ,\lambda )$, where $\psi$ is the target and $\lambda$ a nuisance parameter, a fully ancillary statistic has a law independent of both. A partial ancillary statistic for $\psi$ may have a law depending on $\lambda$ but not on $\psi$. To eliminate $\lambda$ by conditioning, one needs the actual conditional law of the informative statistic to be free of $\lambda$; partial ancillarity by itself does not guarantee this. For example, if independent counts have Poisson distributions with means $\lambda \psi$ and $\lambda (1-\psi )$, their total $N$ has Poisson distribution with mean $\lambda$, independent of $\psi$. The first count conditional on $N=m$ has binomial distribution with parameters $(m,\psi )$. Conditioning removes the nuisance intensity $\lambda$ exactly. More generally, nuisance-free pivotal quantities or conditional likelihoods can be used where available; there is no universal conditioning device that removes every nuisance parameter.

For a concrete example of non-uniqueness of maximal ancillary statistics, consider one observed pair $(U,V)\in \{0,1{\}}^2$, withBoth $U$ and $V$ have Bernoulli distribution with success probability $1/2$, independent of $\theta$, so each is an ancillary statistic. Their generated partitions are different and incomparable. Moreover, each is maximal: refining either two-point cell singles out a point whose probability is $\theta /2$ or $(1-\theta )/2$, which is parameter-dependent. Their joint statistic is not ancillary, since $P_{\theta }(U=V)=\theta$. Thus there is no common finer ancillary partition that reconciles the two choices. The conditional laws differ as labelled experiments: $P_{\theta }(V=U\mid U)=\theta$ and $P_{\theta }(U=V\mid V)=\theta$, but they condition on different observed information. This illustrates why “condition on a maximal ancillary” need not specify a unique procedure.

For the location-scale family, write $Y_i=\mu +\sigma Z_i$, where the $Z_i$ are independent with the fixed density $f_0$. Let $s_Z^2=n^{-1}{\sum }_i(Z_i-\bar{Z}{)}^2$. The transformation givesand thereforeFor $n\ge 2$, $s_Z>0$ almost surely, because an independent sample from a density has probability zero of all observations coinciding. The right side depends only on a sample from the fixed base law, so the entire vector $A$ has a law independent of $(\mu ,\sigma )$: $\boxed{A\text{ is ancillary}.}$ No finite population mean or variance is needed for this algebraic argument. With one observation the displayed standardization is undefined, so at least two observations are implicitly required.

The pair $(\hat{\mu },\hat{\sigma })$ together with $A$ reconstructs the full sample through $Y_i=\hat{\mu }+\hat{\sigma }{A}_i$. Thus conditional inference in a location-scale family useswhere $a$ is the observed residual configuration. Equivalently one can use the parameter-free conditional law of $((\hat{\mu }-\mu )/\sigma ,\hat{\sigma }/\sigma )$ given $A=a$. To see this explicitly, the centered residual subspace has dimension $n-1$, so its radial coordinate $t=\hat{\sigma }$ contributes a Jacobian determinant factor proportional to $t^{n-2}$. The conditional density of $\hat{\mu }=m$, $\hat{\sigma }=t>0$ is therefore proportional toSetting $u=(m-\mu )/\sigma$ and $v=t/\sigma$ gives a conditional density proportional to $v^{n-2}{\prod }_i{f}_0(u+v{a}_i)$, with no unknown parameter. The normalizing factor can depend on $a$, but not on $\mu$ or $\sigma$. These statements concern almost every attainable ancillary value.

Statistical inference uses observed data and a statistical model to learn about unknown features of the process that generated them. In a parametric statistical model, write $X\sim P_{\theta }$: the probability distribution depends on an unknown statistical parameter $\theta$. An estimand specifies the target, such as a mean or a difference of means. A statistic is a function of the observations; an estimator is a statistic used to estimate the target. Uppercase $X$ denotes the random data before observation, while lowercase $x$ denotes their realized values.

Three questions lead to different outputs. Statistical estimation asks which target value the observations suggest. A confidence interval asks for a range produced by a procedure with calibrated coverage probability. A hypothesis test asks whether the observations depart sufficiently from a specified null hypothesis to enter its rejection region. These outputs complement one another: an estimated effect size describes magnitude, while a standard error describes the sampling distribution of an estimator. A p-value describes a tail probability under the null hypothesis.

The appeal of statistical inference is that it turns variable observations into reproducible uncertainty statements. In the normal mean confidence interval with known variance below, a sample mean of $1.2$ has standard error $0.5$: both the estimated shift and its precision matter. The calibration depends on the statistical model, including independence and the specified probability distribution. Increasing sample size improves precision under those assumptions; it does not repair a systematically misleading sampling procedure. Bayesian inference provides another framework, using a prior distribution and a likelihood function to obtain a posterior distribution.