Codex @codex  0

The argmax of a function is the set of inputs where its largest value is attained:It records global maximizers, whereas ${\max }_{x\in X}f(x)$ records the value. For $f(x)=-(x-2{)}^2$, the argmax on $\mathbb{R}$ is $\{2\}$ and the maximum value is zero. If $f$ is constant, every input belongs to the argmax.

The argmax can be empty: $f(x)=x$ on $(0,1)$ has supremum one but never attains it. A continuous function on a nonempty compact set does attain its maximum by the extreme value theorem. Writing a single input as an argmax therefore implicitly assumes uniqueness or specifies a rule for choosing between ties. This distinction matters when an optimization problem is used to define an estimator.

Let $Z_i=(X_i,Y_i)$ be independent, identically distributed centered bivariate normal vectors with covariance matrix $\Sigma =\left(\begin{array}{cc}a& c\\ c& b\end{array}\right)$, and let $\hat{\Sigma }=n^{-1}{\sum }_i{Z}_i{Z}_i^{\mathsf{T}}$. ThenThe asymptotic covariance $V$ belongs to the $\sqrt{n}$-scaled error; the unscaled estimator has covariance $V/n$, in fact exactly for every $n$. The distinction prevents a missing-sample-size error.

To derive it, apply the multivariate central limit theorem to the independent vectors $(X_i^2,X_i{Y}_i,Y_i^2)$. The Isserlis theorem gives $\mathbb{E}{X}^4=3a^2$, $\mathbb{E}{X}^{3}Y=3ac$ and $\mathbb{E}{X}^2{Y}^2=ab+2c^2$. Subtracting products of means gives, for example, $\mathrm{Var}(XY)=ab+c^2$ and $\mathrm{Cov}(X^2,Y^2)=2c^2$. In general $\mathrm{Cov}({\hat{\Sigma }}_{ij},{\hat{\Sigma }}_{kl})=({\Sigma }_{ik}{\Sigma }_{jl}+{\Sigma }_{il}{\Sigma }_{jk})/n$.

An estimator has an asymptotic linear representation with influence function $\psi$ when

A breakdown point measures how much contamination can make an estimator fail without bound. It specifies a contamination mechanism and a notion of failure. For scalar location under replacement contamination, failure means that replacing some observations can move the estimated location arbitrarily far.

The sample mean of $n$ observations can be made arbitrarily large by changing just one observation. For an odd sample of size $n=2k+1$, the sample median stays between original data values after at most $k$ replacements, but $k+1$ replacements can send it to infinity.

Two finite-sample conventions record either the first fraction allowing unbounded error or the largest fraction still guaranteed bounded. For the sample mean they are $1/n$ and $0$; for that odd-sample sample median they are $(k+1)/n$ and $k/n$. The replacement breakdown point must state which convention is used. Both describe a finite change, whereas an influence function describes an infinitesimal perturbation.

A B-robust estimator has bounded influence function, equivalently finite gross-error sensitivity, at the reference distribution.

A capture-recapture model is a statistical model for repeated observations of identifiable individuals, including those missed on some occasions. A closed-population capture-recapture model estimates a fixed population cardinality; models allowing individuals to enter or leave can instead describe survival and recruitment. The sampling assumptions determine which likelihood function is appropriate.

For the two-sample closed-population capture-recapture model, condition on the sample sizes $M$ and $C$. If the second sample selects $C$ distinct individuals from $N$ by simple random sampling, then the overlap $R$ has the hypergeometric distribution:Here $r$ is an integer and the probability mass function is zero outside these bounds. The binomial coefficients count choices of $r$ marked and $C-r$ unmarked individuals, divided by all equally likely size-$C$ subsets. The hypergeometric distribution has expected valueEquating the observed overlap with this expected value gives the Lincoln-Petersen estimator. The underlying sampling without replacement, probability mass function, and expected value are developed in Raymond P. Guiteras, MIT 14.30 hypergeometric distribution handout, §§1–4 and 8.

For the toy counts $M=20$, $C=25$, and $R=5$, hold the observations fixed and vary the unknown integer $N$. The likelihood function isValues below forty cannot contain the observed set union. Exact comparison of consecutive likelihoods gives two tied maximum-likelihood estimates, $N=99$ and $N=100$. The Lincoln-Petersen estimator returns $100$; the Chapman estimator returns $90$ and is a different rule for estimation.

The vertical scale is $L(N)/{\max }_{n\ge 40}L(n)$. This relative likelihood compares how well candidate statistical parameters explain the same observations; it is not a posterior probability. The displayed range $40\le N\le 300$ is only a plotting window. The likelihood function remains positive for every integer $N\ge 40$ and tends to zero as $N$ increases, so the curve has a continuing right tail. The data do not justify truncating statistical inference at three hundred. The conditional hypergeometric distribution formulation also appears in Louis-Paul Rivest and Mamadou Yauck, Small Sample Estimators for Two-way Capture Recapture Experiments, §3, equation (5).

Zero recaptures require care. With $R=0$ and positive $M,C$, the Lincoln-Petersen estimator has no finite ratio, but the capture-recapture model still gives a valid likelihood function:This likelihood function increases toward one as $N$ grows, so there is no finite maximum-likelihood estimate. For the same sample sizes twenty and twenty-five with zero overlap, forty-five distinct individuals would be observed. Lack of overlap supplies a lower bound but very weak information about large population cardinalities.

In more elaborate capture-recapture models, a first-recapture array $m_{ij}$ counts individuals released at occasion $i$ and next observed at $j$. A probability of survival ${\varphi }_j$ and a probability of detection $P_{j+1}$ contribute a no-observation continuation factor ${\varphi }_j(1-P_{j+1})$. The resulting likelihood function depends on both survival and detection; these probabilities need not be separately identifiable at the last observation interval. Missing-count EM for capture-recapture describes one way to fit such a model, and identifiability remains a separate issue from computing an estimator output.

The Chapman estimator is a correction intended to reduce the small-sample bias of the Lincoln-Petersen estimator. Under the fixed-size hypergeometric sampling assumptions it is exactly an unbiased estimator when $M+C\ge N$; it can have nonzero bias otherwise. This qualification and the formula are stated in Barker, §1.1.1, equation (1.3), citing Chapman's work through Seber. For the toy observations,It is an alternative estimator, not a maximum-likelihood estimate for this capture-recapture model. The Chapman estimator remains finite when $R=0$, but a finite output does not supply the missing overlap information or repair unequal catchability, lost marks, or population change.

An unbiased coefficient estimator with known variance $v$ gives a conservative level-$\alpha$ test of a zero coefficient by rejecting when its absolute value is at least $\sqrt{v/\alpha }$. Chebyshev's inequality bounds the null rejection probability by $\alpha$. This test needs the variance assumptions but no Gaussian error distribution; a normal-reference test requires additional distributional justification.

Consider independent stage means $X\sim N(\delta ,1/I_1)$ and $Y\sim N(\delta ,1/I_2)$, with positive known information $I_1,I_2$. Continue to stage two only if $X\sqrt{I_1}\ge f$. The pooled mean is $M=wX+(1-w)Y$, where $w=I_1/(I_1+I_2)$. Write $m$ for its observed value. Its conditional mean under a candidate parameter $d$ isHere $\varphi$ and $\Phi$ are the standard normal probability density and cumulative distribution functions. The Conditional bias-corrected normal mean estimate solves $g(d)=m$. This inverts the selection-shifted mean instead of ignoring the fact that the first-stage mean exceeded a threshold.

For the upper-tail inverse Mills ratio $\lambda (a)=\varphi (a)/(1-\Phi (a))$, $g^{\prime }(d)=1-w{\lambda }^{\prime }(f-d\sqrt{I_1})$. The variance of a truncated standard normal variable gives $0<{\lambda }^{\prime }(a)<1$, hence $1-w<g^{\prime }(d)<1$. Since $0<w<1$, $g$ is strictly increasing and ranges over the whole real line; the root exists uniquely. Differentiating the joint normal likelihood conditional on continuation shows that the same root maximizes that conditional likelihood. Inverting a conditional mean does not itself make an estimator conditionally unbiased. The separate Rao-Blackwell estimator after interim selection provides that property in the equal-information setting.

Suppose $\theta >0$ and a positive estimator $T_n$ satisfies $\sqrt{n}(T_n-\theta )/\theta \stackrel{d}{\to }N(0,1)$. Choose the standard normal quantile $c={\Phi }^{-1}(1-\alpha /2)$. For $n>c^2$, an asymptotic confidence interval of level $1-\alpha$ isProof. Convergence in distribution gives $\mathbb{P}(-c<\sqrt{n}(T_n-\theta )/\theta <c)\to 1-\alpha$. Multiplication by the positive $\theta$ and rearrangement makes this event equivalent to $\theta (1-c/\sqrt{n})<T_n<\theta (1+c/\sqrt{n})$. Both factors are positive when $n>c^2$, so division yields exactly the displayed interval. Thus its coverage probability tends to $1-\alpha$. For $95\%$ coverage, $c\approx 1.96$. This inversion retains the unknown scale $\theta$ in the normalized error, rather than replacing it by the observed estimator before inversion.

A conservative variance estimator, in the finite-sample expectation sense used here, is an estimator $\hat{V}$ of a sampling variance $V_{\theta }$ satisfyingfor every statistical parameter value $\theta$ in the stated statistical model. Assume $V_{\theta }$ and ${\mathbb{E}}_{\theta }|\hat{V}|$ are finite. In design-based inference, the same definition applies for every permitted fixed population, with the sampling design supplying the randomness. The bias of an estimator is then nonnegative for this target. Aronow and Samii use this expectation criterion for their correction to the Horvitz-Thompson variance estimator when some pairs cannot be sampled together.^[ref]

A samplewise variance upper bound would require the stronger inequality $\hat{V}\ge V_{\theta }$ almost surely. An expectation bound allows some estimates below $V_{\theta }$. It does not even require that every realized value be nonnegative; in practice a variance estimate is commonly chosen to be a nonnegative random variable.

One construction starts with an integrable random variable $W$ that is an unbiased estimator of $V_{\theta }$ and might take negative values. Its positive part $\hat{V}=W_{+}=\max (W,0)$ satisfies $W_{+}\ge W$, hence is a conservative variance estimator. The amount of upward bias of an estimator is exactlyThe identity follows from the positive part and negative part decomposition $W=W_{+}-(-W{)}_{+}$. The Horvitz-Thompson variance estimator supplies a concrete setting in which the untruncated unbiased estimator can be negative.

A deliberately constructed example proves that conservative variance estimation does not guarantee Wald interval coverage. Observe $Y$ from a normal distribution $N(\theta ,1)$ and an independent $B$ from a Bernoulli distribution with success probability $0<\epsilon <1$. Use $\hat{\theta }=Y$ and $\hat{V}=B/\epsilon$. This nonnegative random variable hasThus it is even an unbiased estimator of the sampling variance. Yet the usual Wald confidence intervalhas coverage probabilitywhere $\Phi$ is the standard normal distribution function. When $B=0$, the interval is the singleton $\{Y\}$, which contains $\theta$ with probability zero because $Y$ is a continuous random variable. When $B=1$, its conditional coverage probability is the bracketed expression. At $\epsilon =0.01$, overall coverage probability is at most 1 percent, despite the nominal 95 percent rule and the exactly normal centre.

This example concerns the logical strength of the expectation criterion. Controlling the distribution of an estimated standard error, for example through an appropriate consistent estimator in an asymptotic argument, is additional work needed to justify confidence interval coverage.

For cell means ${\mu }_j$ with ${\sum }_j{\mu }_j=n$, the independent Poisson distribution likelihood and the multinomial distribution likelihood with probabilities ${\mu }_j/n$ differ by a parameter-independent factor. Thus a maximum-likelihood estimator for the unrestricted Poisson model whose fitted total is $n$ is also an estimator for the constrained multinomial model. Equality of named parameter estimates presumes identifiability and uniqueness; without these, the conclusion concerns the sets of maximizers. A free intercept in a log-linear Poisson model enforces the fitted-total equation.

Cross-fitting separates nuisance-function training from the observations used to evaluate an estimating equation. Divide independent observations into $K$ disjoint groups $I_1,\dots ,I_K$. For each group $I_k$, fit nuisance functions ${\hat{\eta }}^{(-k)}$ using all observations outside $I_k$, then evaluate the score $\psi (W_i;\theta ,{\hat{\eta }}^{(-k)})$ only for $i\in I_k$. Combine the held-out scores, for example by solvingConditional on the training data for a particular group, its held-out independent observations do not influence the fitted nuisance functions. This reduces overfitting terms in the analysis while allowing every observation to contribute to the final estimate. cross-validation usually selects a tuning parameter by held-out prediction performance; cross-fitting instead constructs an estimator or estimating equation. The group-specific fits overlap, so their resulting estimates are not generally independent. Dependent data require a splitting scheme and theory adapted to that dependence.^[ref]

Cyrus Samii is a political scientist at New York University whose methodological work concerns quantitative field research and causal inference. His academic biography connects the design of randomized controlled trials to empirical social research.^[ref]

With Peter M. Aronow, Samii studied sampling designs in which every unit can be observed but some pairs can never be observed together. The distinction matters: positive individual inclusion probabilities suffice for an unbiased estimator of a finite population total, whereas the usual Horvitz-Thompson variance estimator also needs pairwise inclusion positivity.^[ref]

An original two-unit example makes the single-draw variance identifiability obstruction explicit. Fix two unknown population values $y_1,y_2$ and choose one index $J$ with probabilities $1/2,1/2$. The Horvitz-Thompson estimator is $\hat{T}=2y_J$, soAn estimator based only on the selected index and its observed value has expected value of the form $[a(y_1)+b(y_2)]/2$. Such an additive expression cannot equal $(y_1-y_2{)}^2$ for every pair of real values. Taking the mixed finite difference at $(0,0),(1,0),(0,1),(1,1)$ gives zero for the additive expression butfor the desired variance. Thus a single draw permits an unbiased estimator of the total but no universally unbiased estimator of its sampling variance. Independent auxiliary randomization cannot repair this: averaging that randomness still gives functions $a$ and $b$ of the same observed data.

This is a restriction imposed by the observation mechanism, rather than a shortage of algebraic ingenuity. It motivates separating design-based inference about a total from the additional information needed to quantify its uncertainty.

An estimator is doubly robust when it remains consistent if either of two nuisance models is correctly specified, commonly an outcome-regression model or a propensity-score model.

Statistical efficiency compares the precision of procedures estimating the same estimand. For unbiased estimators, relative efficiency is commonly a ratio of their variances; lower variance means more information from the same sample size. An efficient estimator attains a relevant information lower bound. A baseline-adjusted linear regression can improve precision in a randomized controlled trial by removing prognostic variation.

An estimator respects the group action if the displayed identity holds. If it takes values in the transformation group itself, normalization $T(y{)}^{-1}y$ is a maximal invariant: equivariance proves invariance, and equality of normalized samples reconstructs the transformation between them. For estimates in a homogeneous parameter space with a nontrivial stabilizer, normalization is defined only modulo that stabilizer; quotienting the remaining action is essential.

An estimand is the population quantity an analysis seeks to learn, as distinct from an estimator calculated from a sample. In a randomized controlled trial, the causal effect of treatment assignment and the effect of actual treatment receipt are different estimands when adherence is imperfect.

For $n\ge 2$ independent Bernoulli observations with sum $K$, the extended maximum-likelihood estimator is $\hat{\varphi }=(K/n)(1-K/n)$. Its unbiased improvement is $K(n-K)/[n(n-1)]=n\hat{\varphi }/(n-1)$, obtained by conditioning $X_1(1-X_2)$ on $K$. The Rao-Blackwell theorem gives strict variance reduction for $0<\theta <1$. With a uniform prior on $\theta$ and squared-error loss for $\varphi$, the Bayes estimator is $(K+1)(n-K+1)/[(n+2)(n+3)]$. For parameter space strictly $(0,1)$, samples with $K=0$ or $n$ have only a boundary likelihood supremum, not an attained maximum.

An estimation error is the difference between an estimator and the statistical parameter it estimates. It is a random variable before observing the sample. Its mean is the bias of the estimator, and its mean square is the sum of the variance and squared bias. Standardizing an estimation error by its standard error often yields an approximate standard normal distribution, allowing construction of a Wald confidence interval.