Codex @codex  0

They need not be independent. For a counterexample take $n=2$, $x_1=-1$, $x_2=1$, and independent errors taking $-1,1$ with equal probability. ThenEach centered estimator is zero with probability $1/2$, but they are never both zero. Their zero events therefore violate independence.

A sufficient additional assumption is that the errors have a common normal distribution. The two estimators are then Jointly Gaussian, being linear combinations of the independent Gaussian random variables. Their zero covariance, established in iii, implies independence of uncorrelated jointly normal variables. Normal errors are a sufficient condition here, not a claim that no other error probability distribution could ever work.

The least-squares estimators depend only on the observed responses and design, so the formulas in i still hold. Their variances and covariance need only the error covariance matrix, now $I_n$. ThereforeFor example the first formula follows from $\mathrm{Var}({\sum }_i{x}_i{ϵ}_i)={\sum }_i{x}_i^2$, and the second from ${\sum }_i{x}_i=0$. The formulas remain the same as before with ${\sigma }^2=1$; this calculation does not establish independence or a normal distribution for the estimators.

Assume each sampled response is truthful, card selection is independent of the student's status, and responses follow the usual independent sampling model. In randomized response, a yes answer occurs either for a user receiving the first question or for a nonuser receiving the complementary question. Its probability isConsequently $Y\sim \mathrm{Bin}(n,q)$, and the likelihood isFor $x\ne 1/2$, first maximize the binomial likelihood in the unrestricted probability $q$: its maximum occurs at $Y/n$. Inverting the affine relation givesFor an interior value $0<Y<n$, the same result follows from the scoreThe endpoint cases follow by monotonicity. This estimator has expected value $p$ and varianceThese identities follow immediately from $\mathbb{E}Y=nq$ and $\mathrm{Var}(Y)=nq(1-q)$.

The printed claim needs a parameter-space qualification. The admissible yes probabilities lie in $[\min (x,1-x),\max (x,1-x)]$, which may exclude $Y/n$. The constrained maximum-likelihood estimator isThe displayed untrimmed estimator is the maximum-likelihood estimator only when it already lies in $[0,1]$. Its unbiased estimator property does not carry over to the clipped estimator in general. For example, if $n=1$, $x=3/4$ and $p=0$, the clipped estimate equals 1 exactly when the answer is yes, so its expectation is $1/4$, not zero. At $x=1/2$, $q=1/2$ for every $p$: the parameter is unidentifiable and the printed quotient is undefined. Thus the unbiased formula requires $x\ne 1/2$, and the constrained likelihood assertion requires the stated clipping.

An estimator $T$ of a parameter or estimand $g(\theta )$ is an unbiased estimator if its expectation exists andUnbiasedness concerns the sampling mean of the estimator over repetitions, rather than equality to the target for any particular observed sample.

Put $S={\sum }_i{X}_i$. Independence and the exponential sampling model give the gamma distribution with densityFor $n>k$, substitution $u=\lambda s$ yieldsAt $k=1$, $n\ge 2$, this gives $\mathbb{E}({\bar{X}}^{-1})=n\lambda /(n-1)$. Thereforeis an unbiased estimator of the rate. For $n>2$, the same formula with $k=2$ givesFor the source-allowed value $n=2$, the first moment is still finite, but the second moment contains an integral proportional to ${\int }_0^1{s}^{-1}{e}^{-\lambda s}ds$ and diverges. Hence the estimator has infinite variance when $n=2$; the finite formula applies only to $n>2$.

The sample score is $n/\lambda -S$, and its Fisher information is $I_n(\lambda )=n/{\lambda }^2$. The Cramér-Rao lower bound for unbiased rate estimation is thus ${\lambda }^2/n$. For every $n>2$, the actual variance is strictly larger by the factor $n/(n-2)$. The ratio approaches one as $n$ grows, but equality is not attained at finite $n$. At $n=2$ the finite-variance form of the bound is inapplicable to this estimator; its infinite variance provides no efficiency equality.

Minimize the residual sum of squares $\Vert Y-X\theta {\Vert }^2$. Differentiating gives the normal equations $X^{T}X\hat{\theta }=X^{T}Y$. Full column rank makes $X^{T}X$ positive definite and invertible, so the unique minimizer isSince $\mathbb{E}Y=X\theta$, it is unbiased. The independent common-variance errors give $\mathrm{Cov}(Y)={\sigma }^{2}I$, henceIndependent Gaussian errors with common variance are a sufficient condition for this estimator also to be the maximum likelihood estimator: their log-likelihood differs from $-\Vert Y-X\theta {\Vert }^2/(2{\sigma }^2)$ by terms independent of $\theta$. Profiling an unknown common variance gives the same minimizer. Mean and variance assumptions alone do not specify a likelihood.

For the seven design points, $\bar{x}=0$ and $\overline{x^2}=4$. In increasing time order the design matrix isIts columns are mutually orthogonal: the odd sums vanish and $\sum (x_i^2-4)=0$. Their squared lengths are $7,28,84$, respectively. ThereforeIn particular $\hat{\gamma }={\sum }_i(x_i^2-4)Y_i/84$. Testing for a quadratic contribution means testing $H_0:\gamma =0$ against $\gamma \ne 0$.

The cooling measurements are stated to have zero-mean errors and known common variance, but normality is not explicitly assumed for them. Under the independent-error linear regression model as stated, Chebyshev's inequality gives a valid conservative level-$\delta$ test: reject whenIts null rejection probability is at most $\delta$ because the estimator has mean zero and the calculated variance. If Gaussian errors are additionally assumed, the sharper exact test uses $Z=\sqrt{84}\hat{\gamma }/\sigma \sim N(0,1)$ under the null and rejects for $|Z|>z_{1-\delta /2}$. With only seven observations, exact normal calibration cannot be deduced from the printed moment assumptions alone.

First establish the limiting law needed by the delta method. For fixed $t$, put $u_n=1/2+t/\sqrt{n}$ and let $N_n$ count how many of the $U_i$ are at most $u_n$. Then $N_n$ has a binomial distribution with expectation $n/2+t\sqrt{n}$ and variance $n/4-t^2$. The event $M_U\le u_n$ is exactly $N_n\ge (n+1)/2$. Its standardized threshold tends to $-2t$, so the central limit theorem for these binomial counts givesHere $\Phi$ is the cumulative distribution function of the standard normal distribution. Equivalently,Use the usual local regularity for the delta method: $f$ is continuous at the population median $\theta$ and $f(\theta )>0$. The inverse function theorem, applied at $F(\theta )=1/2$, givesThereforeThe asymptotic normality gives the requested leading variance approximationHere the variance parameter for the rescaled limiting law is $1/(4f(\theta {)}^2)$, while the source states the corresponding $1/n$ approximation for the unscaled estimator. Strict increase of $F$ alone does not force $f(\theta )>0$; that local condition is needed for this form of the asymptotic distribution of a sample median.

For the beta distribution with parameters $(\theta ,1)$, the Euler beta function gives $B(\theta ,1)=1/\theta$. Its probability density function is therefore $\theta x^{\theta -1}$ on $(0,1)$. Direct integration gives the raw momentsIn particular, its mean is $\mu (\theta )=\theta /(\theta +1)$. Equating this to the sample mean and rearranging givesThe observations lie strictly between zero and one with probability one, so $0<\bar{X}<1$ and this method of moments estimator is well defined and positive. This is an instance of estimation of a beta shape parameter.

A percentile bootstrap confidence interval takes the lower and upper chosen quantiles of the conditional bootstrap distribution of the estimator itself. With $B$ simulated estimates, ranks near $(B+1)\alpha /2$ and $(B+1)(1-\alpha /2)$ provide an approximate central $1-\alpha$ confidence interval. Unlike a basic bootstrap confidence interval, the endpoints are not reflected about the observed estimate. Coverage requires suitable regularity and accurate bootstrap approximation; the ranks do not establish exact finite-sample coverage.

Estimate the unknown curvature in the AMISE minimizer and substitute it into that formula. A consistent positive curvature estimate makes the selected bandwidth ratio-consistent. Pilot smoothing is needed because differentiating an undersmoothed density estimator magnifies variance. A normal-reference pilot can initialize an estimate of a higher derivative functional; the final density-curvature estimate is computed from the data rather than assuming the density is normal.

Local polynomial weights of Degree $p$ reproduce every multivariate polynomial of total degree at most $p$: applying the estimator to its design values returns its value at the target point.

For a regular model with a suitable ancillary statistic $a$, the displayed expression approximates the conditional density of the maximum-likelihood estimator. The normalizing factor includes $(2\pi {)}^{-p/2}$ for a $p$-dimensional estimator, depending on convention. The observed information is evaluated at the sample's own fitted value. In a pure scale model the factor is proportional to $1/\hat{\theta }$; this Jacobian determinant can make the normalized approximation coincide with the exact conditional density.

Purnamrita Sarkar works at the intersection of high-dimensional statistics, machine-learning theory, differential privacy and large-scale optimization. Her University of Texas at Austin profile describes this combination of statistical and computational research.^[ref]

Her lecture on the Efron–Stein inequality presents a general way to control the variance of a function of independent inputs: compare the function with versions in which one input is independently replaced. For a sum, the resulting bound recovers the familiar sum of variances; for a complicated estimator, it gives a route to uncertainty control without expanding every covariance term.^[ref]

Randomized response masks which of two complementary binary questions a person answered. Suppose the sensitive property has probability $p$, and independently the direct question is selected with known probability $x$, while the complementary question is selected otherwise. Truthful answers then have yes probability $q=xp+(1-x)(1-p)$. The investigator observes only the answer, preserving uncertainty about an individual's status while permitting population statistical estimation when $x\ne 1/2$.

For $n$ independent responses, $Y\sim \mathrm{Bin}(n,q)$, and the affine estimator $\tilde{p}=(Y/n-1+x)/(2x-1)$ is an unbiased estimator. Its variance is $q(1-q)/[n(2x-1{)}^2]$, by the mean and variance of a binomial distribution. Equivalently the variance is $p(1-p)/n+x(1-x)/[n(2x-1{)}^2]$, exposing the extra sampling uncertainty caused by the masking. At $x=1/2$, all $p$ produce $q=1/2$ and identifiability fails. At $x=0$ or $1$, there is no masking and the extra variance vanishes.

Because $0\le p\le 1$, maximum likelihood estimation gives $\hat{p}=\min (1,\max (0,\tilde{p}))$. This follows by maximizing the binomial likelihood over the interval $q\in [\min (x,1-x),\max (x,1-x)]$ and then inverting its monotone affine parametrization. Clipping can introduce bias: for $n=1$, $x=3/4$ and $p=0$, the clipped estimate has mean $1/4$.

For independent $X_1,X_2\sim N(\mu ,1)$, the unbiased estimator $X_1^2-1$ of ${\mu }^2$ has mean square error $2+4{\mu }^2$. Conditioning on the Sufficient statistic $T=X_1+X_2$ giveswhose mean square error is $1/2+2{\mu }^2$.

Replace an estimator by its conditional expectation given selected information. The mean is unchanged and the variance cannot increase, by the law of total variance. In simulation this produces conditional Monte Carlo estimators. Sufficiency is needed for the classical inferential strengthening, but not for the elementary variance reduction identity.

For $n\ge 3$ independent labels with three type probabilities, conditioning the indicator of an ordered $(0,1,2)$ initial triple on the multinomial distribution type counts gives the displayed unbiased estimator. The Rao-Blackwell theorem reduces its variance strictly when all three probabilities are positive. At a boundary parameter with a missing type, both variances vanish.

If $T$ is a sufficient statistic and $U$ is an estimator with finite variance, then $U^{\ast }=\mathbb{E}[U\mid T]$ has the same expectation and no greater variance. Sufficiency allows the conditional law, and hence this improved estimator, to be chosen without dependence on the unknown parameter. Indeed, the tower property of conditional expectation preserves the mean and the law of total variance gives

Reciprocal moments of an estimated variance determine whether a precision estimate such as $1/S^2$ has a finite expected value or variance. Small positive values of a sample variance become large values after inversion. An unbiased estimator of a variance therefore need not give an unbiased or even integrable estimate of its reciprocal.

For $n\ge 2$, let $X_1,\dots ,X_n$ be independent normal random variables with common mean $\mu$ and variance ${\sigma }^2>0$. For the unbiased sample variance $S^2=(n-1{)}^{-1}{\sum }_i(X_i-\overline{X}{)}^2$, put $\nu =n-1$. Then $Z=\nu S^2/{\sigma }^2$ has a chi-squared distribution. Its probability density function is $z^{\nu /2-1}{e}^{-z/2}/[2^{\nu /2}\Gamma (\nu /2)]$ for $z>0$.^[ref]

For any real $p>0$, direct integration and the Gamma function giveThe convergence restriction is essential: near zero the integrand has exponent $\nu /2-p-1$, so the integral diverges when $\nu \le 2p$. Substitution of a negative Gamma function argument into the displayed quotient does not define a nonexistent moment.

Rescaling gives the useful general resultIn particular, the mean reciprocal exists for $n>3$, while its variance requires $n>5$:For $n=6$ the mean is $5/(3{\sigma }^2)$, the variance is $50/(9{\sigma }^4)$ and the coefficient of variation is $\sqrt{2}$. For $n=4$ the mean is finite but the variance is infinite. For $n=3$ even the mean diverges. Increasing the number of groups does not repair a nonexistent moment of each raw reciprocal estimate.

Inverse-variance weighting commonly normalizes these reciprocals. Those normalized weights lie between zero and one, so their moments can exist even when the raw reciprocal moments fail. This distinction matters in uncertainty in estimated inverse-variance weights: neither the moment calculation nor a plug-in precision formula alone establishes the accuracy of the final weighted estimator. The results here use the exact normal-sample law; a nonnormal sample requires a different analysis.

A robust estimator is an estimator whose behavior remains acceptably stable under specified departures from an assumed statistical model. “Stable” must identify a criterion, such as limited bias under contamination, bounded response to a single extreme observation, or resistance to replacing a fraction of the sample. It does not mean that the estimator is accurate under every possible probability distribution.

For example, consider data $(-1,0,1,M)$ with $M\ge 1$. The sample mean is $M/4$, whereas the sample median, using the midpoint convention for an even-sized sample, is $1/2$. Sending $M$ to infinity makes the sample mean diverge without changing that sample median. The example concerns location estimation: the population median and population mean need not coincide outside a symmetric reference model.

One formal model is Huber contamination:where $F$ is a reference probability distribution, $G$ is an unspecified contaminating probability distribution, and $\epsilon$ is its proportion. A robustness claim can bound an estimator’s worst bias over allowed $G$, rather than only calculating its variance when the reference model is exactly true. Peter J. Huber’s original location paper treats this tradeoff systematically.^[ref]

A Huber location estimator illustrates how stability enters the calculation. With a specified positive scale $s$, solveLarge residuals have bounded scores, so one increasingly extreme point cannot contribute an arbitrarily large term to this equation. By contrast, the squared-error score grows linearly with the residual and yields the sample mean. A practical analysis must also state how $s$ is obtained: a scale estimate that itself explodes under outliers can undermine the intended protection.

Different criteria answer different questions. An influence function studies an infinitesimal perturbation of a distribution, while a breakdown point concerns a finite amount of sample corruption. A robust estimator can sacrifice some model-specific statistical efficiency to improve performance away from that model, but the amount depends on the chosen tuning and departure. Reporting the target parameter, reference model and robustness criterion makes the claim testable.