An estimand is the population quantity an analysis seeks to learn, as distinct from an estimator calculated from a sample. In a randomized controlled trial, the causal effect of treatment assignment and the effect of actual treatment receipt are different estimands when adherence is imperfect.
Statistical efficiency compares the precision of procedures estimating the same estimand. For unbiased estimators, relative efficiency is commonly a ratio of their variances; lower variance means more information from the same sample size. An efficient estimator attains a relevant information lower bound. A baseline-adjusted linear regression can improve precision in a randomized controlled trial by removing prognostic variation.
A statistical functional assigns a target value to each probability measure in a statistical model. Examples include an expected value, a quantile, and an integral of a power of a probability density function. Its behavior along statistical paths determines whether first-order influence-function representers are available.
Along bounded density tilts, this functional has derivative . At a bounded baseline density its canonical gradient, relative to these regular paths, is . A common local bound on nearby densities suffices to obtain the same derivative along arbitrary differentiable-in-quadratic-mean paths in that bounded neighborhood. A bounded baseline alone is insufficient, because of the spike obstruction to density-power differentiability.
A perturbation can have negligible Hellinger distance but a large integral of its density's fourth power. At the uniform density on , let on and zero elsewhere, and let . Then and , so the score function is zero. However . Thus the density fourth-power functional is not continuous along this differentiable-in-quadratic-mean path, despite a bounded baseline and individually bounded nearby densities.
A statistical functional is pathwise differentiable relative to chosen statistical paths if its derivative along every path depends only on that path's score function and defines a bounded linear functional on their statistical tangent space. The Riesz representation theorem expresses this derivative as an L2 inner product with a unique element of the statistical tangent space, the canonical gradient. A path family and the associated derivative remainder conditions must both be specified; a formal derivative along one convenient family does not establish differentiability along all paths.
An influence-function representer is a centered square-integrable function representing derivatives of a statistical functional along all admissible score functions. Representers may differ by a function orthogonal to the statistical tangent space. This pathwise definition is distinct from defining an influence function solely by point-mass contamination paths.
The canonical gradient is the influence-function representer in the statistical tangent space . It equals the orthogonal projection onto of any representer. Every other representer differs from it by an element of , so the Pythagorean theorem in an inner-product space gives it minimum variance. For in an unrestricted density model with bounded , it is .
Statistical estimation uses observed data to infer numerical properties of a population or model. A point estimator gives a value, while a confidence interval describes uncertainty with a specified coverage property. Maximum likelihood estimation is one approach; bias, variance, identifiability and the sampling or selection mechanism all affect an estimate's interpretation.
Probability sampling chooses observational units by a specified random mechanism with known inclusion probabilities. This separates design-based representativeness from the convenience of obtaining particular observations. Simple random sampling, stratified sampling and cluster sampling are different designs with different precision and cost properties.
Cluster sampling chooses groups of observational units, such as production days, rather than dispersed individual units. Testing every unit in a selected group is a one-stage cluster sample. Positive within-cluster intraclass correlation reduces the information supplied by a fixed number of sampled units, although collecting clustered observations can be cheaper.
The design effect is a sampling variance under a complex design divided by the corresponding simple-random-sampling variance. For independent equal-size clusters of size with exchangeable intraclass correlation , a common mean-estimation approximation is . It expresses the loss of effective sample size from positive correlation; finite-population and unequal-cluster-size designs require their own calculation.
Stratified sampling partitions a population into strata and samples separately within each. It guarantees specified representation and can improve precision when the strata describe population variation. Equal sampling fractions give equal unit inclusion probabilities; unequal fractions require appropriate weighting for population summaries.
A fixed-size simple random sample gives every subset of the required size the same selection probability. Taking the first labels from a uniformly random permutation of labels produces this design. Restricting a larger uniform permutation to the desired label set preserves uniformity; biased modulo mappings should not replace uniform selection.
A scoring rule assigns a reward or loss to an announced probability distribution and realized outcome. In the reward convention, a proper scoring rule makes truthful reporting maximize expected reward; the loss convention reverses the comparison.
For a binary outcome, rewarding the reported probability of the realized outcome gives expected score under genuine probability . The optimal report is an endpoint whenever , so this is not a proper scoring rule.
Truthful reporting is an optimizer of expected reward for every admissible true distribution. Propriety matters because it aligns a forecaster's reward with an honest expression of uncertainty.
A proper scoring rule is strictly proper when truthful reporting is the unique optimizing distribution. For densities, equality is understood almost everywhere relative to the reference measure.
Under true density , the advantage of truthful reporting is the Kullback-Leibler divergence . Its nonnegativity and equality condition prove strict propriety when the expected log scores are well defined.
The probability chain rule makes a sum of sequential predictive log scores equal the log Bayesian model evidence when the forecasts are coherent Bayesian one-step predictions under proper priors. Differences of total scores are therefore log Bayes factors.
The number of free dimensions remaining after specified constraints often determines a reference sampling distribution. In a full-rank normal linear model with observations and mean coefficients, the residual space has dimension . A regular nested likelihood-ratio test instead uses the difference in free statistical parameter dimensions. Boundary constraints may invalidate this regular calibration.
A statistic is a measurable function of observed data whose definition does not involve an unknown parameter. It may be scalar or vector valued. Sufficient statistics summarize all information about a parameter in the conditional-distribution sense.
A meta-analysis combines statistical evidence from several studies addressing a common estimand. It may assume a shared effect or use a random-effects meta-analysis to represent between-study variation. Inverse-variance weighted means combine independent effect estimates when their variances are known or estimated; differences in study validity and effect modifiers require substantive assessment, not just weighting.
Divide studies by a specified characteristic and estimate an effect in each subgroup, then test the difference using an independent subgroup contrast in meta-analysis or an appropriate interaction model. The test of a between-group difference is distinct from separate tests against no effect within each group. Exploratory splits can be affected by multiple testing and by confounding between study characteristics.
A forest plot shows each study's effect estimate and confidence interval, together with a pooled estimate when appropriate. A vertical null-effect line allows comparison on a common scale. For an odds ratio the null is one and a logarithmic axis treats reciprocal ratios symmetrically; marker areas commonly encode study weights.
A funnel plot displays study effect estimates against a measure of their precision, often standard error increasing down the vertical axis. Under a common-effect model, imprecise estimates spread more widely around the effect. Asymmetry is evidence of a small-study effect and can be consistent with publication bias, but it has other possible explanations.
A small-study effect is a systematic relationship between study precision or size and estimated effect. It may result from selective publication, design differences, effect modification or other heterogeneity. A regression of effect on its standard error tests one such relationship; it does not by itself determine which mechanism generated it.
Publication bias arises when the availability of studies depends on the results, such as their significance, magnitude or direction. A meta-analysis of the available studies can then differ systematically from one including all eligible evidence. Funnel plot asymmetry can motivate investigation but does not establish this mechanism; heterogeneity and other small-study effects can also cause asymmetry.
A selection model combines a distribution for all study results with a probability that a result becomes available. The observed distribution is reweighted by this probability and renormalized. Assumed or estimated selection mechanisms permit sensitivity analysis of a meta-analysis, but unavailable results generally prevent a fully assumption-free correction.
Trim and fill estimates missing studies from funnel plot asymmetry by trimming extreme studies to locate a centre and filling in counterparts on the less represented side. The adjusted meta-analysis includes these imputed estimates. Its interpretation depends on a symmetry model for missing evidence; it is not a guarantee that genuine publication bias has been removed.
For independent subgroup estimates with variances , the null contrast has Wald statistic . If evidence overlaps, include minus twice the covariance in the contrast variance. A nonsignificant comparison is not proof of equivalence.
Systematic displacement of an individual study estimate caused by design, conduct, analysis or selective reporting. Combining estimates can reduce random error without reducing this bias. Bias-adjusted meta-analysis and analyses restricted or stratified by relevant bias domains can assess its impact.
Model study estimates as noisy versions of the target effects plus study-specific biases. External information or sensitivity parameters describe those biases and their uncertainty. Removing an assumed bias without propagating its uncertainty overstates precision; observed study data alone generally do not identify the adjustment.
A common-effect synthesis treats study estimates as estimates of one shared effect. For independent approximately unbiased estimates with variances , use with . This model does not accommodate additional between-study heterogeneity.
For independent two-by-two tables with counts and total , the Mantel–Haenszel pooled odds ratio is the displayed ratio of sums. When all , it is also the weighted arithmetic mean of study odds ratios with weights . These differ from inverse estimated variance weights for log odds ratios. Both constructions can represent a fixed-effect meta-analysis, but they should not be silently interchanged when interpreting a supplied plot.
Refit a meta-analysis after deleting each study, comparing fitted effects, uncertainty and heterogeneity to the full fit. For fixed weights , total weight and weighted effect , direct subtraction of the two weighted means gives . Re-estimating heterogeneity additionally measures influence through the weights. This is a sensitivity diagnostic, not an automatic rule for removing discordant studies.
For independent study estimates with within-study variances , a usual approximate model is and . Its marginal distribution is , so for fixed heterogeneity the inverse-variance weighted mean uses weights . Here describes the mean effect across comparable studies; represents between-study variation. Estimation and uncertainty for heterogeneity are essential, especially with few studies.
A Student t random-effect model assigns Student's t-distributions to exchangeable study effects, allowing heavier tails than a normal distribution hierarchy. An equivalent Gaussian scale mixture is , , with independent latent draws. For the variance is , so is a scale rather than a standard deviation. Small latent precisions weaken shrinkage for atypical studies while retaining partial pooling for the rest.
Variation in the underlying study effects, additional to sampling error. The usual normal random-effects meta-analysis models it by a between-study variance . It can reflect effect modification, design differences or within-study bias; it is not automatically a biological treatment difference.
Meta-regression relates study effect estimates to study-level explanatory variables. A common random-effects meta-analysis extension is , where and . For log odds ratios, is a ratio of underlying odds ratios per unit of . Few studies, correlated design changes and exploratory variable selection limit interpretation; a study-level association does not identify an individual-level causal effect. The interpretation of study-level predictors is also discussed in the Cochrane methods handbook, section 10.11.
For conditionally independent effects , each pair difference has normal distribution . With , an upper bound on ensures, by the union bound, that . Any proper scale prior distribution supported on preserves this bound after averaging. This is conservative simultaneous calibration, rather than the weaker statement about one selected pair. For log odds ratios, a bound corresponds to a ratio-of-odds-ratios bound .
For and studies, . Usually reported as a percentage, it describes excess variation relative to the observed variation on this heterogeneity scale. It is not a proportion of studies with different effects or a proportion of an individual clinical outcome.
For studies let , , , and let be Cochran's Q statistic. The moment estimate is when . It estimates the variance of underlying study effects, not their sampling variance.
With fixed inverse-variance weights and weighted mean , . Under a common-effect normal model with independent estimates and known variances it has a chi-squared distribution with degrees of freedom. Estimated variances make this calibration approximate.
A network meta-analysis jointly compares several treatments using a graph of direct randomized controlled trials and indirect treatment comparisons. If effects are measured relative to a reference treatment, consistency means . A connected graph identifies all relative effects under that model. Transitivity in network meta-analysis is the substantive comparability assumption supporting these relations; inconsistency can be assessed when the network has loops.
For independent studies comparing treatments and to common control , a consistent log odds ratio comparison uses , with variance . This follows from subtracting two estimates of additive effects relative to the same reference. With overlapping evidence, subtract twice their covariance. Causal comparability across the studies is supplied by transitivity in network meta-analysis, not by the variance calculation.
Transitivity requires the studies of different treatment comparisons to be sufficiently comparable in distributions of effect modifiers, outcome definitions and other design features to support an indirect treatment comparison in one target population. For example, a treatment that works differently by disease severity cannot safely be compared indirectly across trials with systematically different severity distributions. Statistical consistency is an implication of suitable transitivity and modeling assumptions, not a substitute for assessing them.
A random set of parameter values with specified repeated-sampling coverage: for every permitted . Exact coverage, asymptotic coverage and pointwise versus uniform validity must be distinguished. A confidence interval is the scalar interval case.
An ellipsoidal confidence region defined by a positive-definite matrix and a calibrated threshold . For a known-covariance multivariate normal distribution sample, and the quantile of the chi-squared distribution with degrees of freedom give exact coverage for the -dimensional mean.
A collection of observed random variables used to infer a statistical model or its parameters. A common assumption is that the observations are independent and identically distributed random variables; dependence must otherwise be included in the model. Its randomness induces the sampling distribution of a statistic.
Semiparametric statistics studies models with a finite-dimensional target parameter and an infinite-dimensional nuisance component.
If the parametric score function has zero conditional expectation given a covariate , it is orthogonal to the centered functions of that form the covariate-density nuisance tangent space. Its efficient score then equals its parametric score function, and the unknown covariate distribution causes no loss of Fisher information. This applies to Gaussian regression scores and more generally to regular conditional models with unrestricted covariate distribution and no additional nuisance components.
The nuisance tangent space is the closed linear span of score functions from statistical paths that vary the nuisance parameter while fixing the target statistical parameter. It is a closed subspace of a Hilbert space inside . Removing its component from a parametric score function leaves the information that nuisance variation cannot imitate.
For independent-error regression with a zero-mean error, bounded density paths must preserve both normalization and the first moment. Their score functions therefore satisfy two constraints. When , truncation followed by two small bounded moment corrections shows these scores are dense in the displayed closed subspace of a Hilbert space. By independence, they are orthogonal to every with . The same constraints apply to other paths only under regularity permitting differentiation of the first moment.
The efficient score is the residual after orthogonal projection of the parametric score function onto the nuisance tangent space. It is centered and orthogonal to every nuisance direction. Its squared L2 norm is the efficient information.
With unknown independent covariate and centered error distributions, take the regular mean-preserving error tangent space, let , and . Under , and finite second moments, orthogonal projection removes from . Thus the efficient score and efficient information are and . The reduction to is valid for a normal distribution of errors or for constant , but need not hold otherwise. Centered covariates and logistic distribution errors give the counterexample .
The decomposition is orthogonal. Taking the inner product with the efficient score gives . Centering of the efficient score follows because the nuisance tangent space and the parametric score function lie in the closed mean-zero L2 space.
The efficient information for a scalar target statistical parameter is the squared L2 norm of its efficient score. It cannot exceed the parametric Fisher information, by the Pythagorean theorem in an inner-product space. It can vanish when first-order target variation can be reproduced by nuisance variation.
A semiparametric estimator targets a finite-dimensional parameter while estimating or otherwise accommodating an infinite-dimensional nuisance component.
A partially linear model combines a linear coefficient with an unrestricted nuisance function, for example .
A statistical functional is Fisher-consistent for a parameter when evaluating it at the model distribution returns that parameter.
A scale estimator measures the spread of a distribution and transforms proportionally when all observations are multiplied by a positive constant.
A statistical estimator is consistent when it converges in probability to the target parameter as the sample size tends to infinity.
A plug-in estimator replaces unknown components of a statistical functional by estimators and evaluates the same functional at those fitted components.
Assouad's lemma lower-bounds minimax risk by embedding a Hamming hypercube of separated parameters whose neighboring statistical experiments are difficult to distinguish.
An Assouad hypercube is a family in which loss separates proportionally to Hamming distance while distributions at neighboring vertices remain close in total variation.
Robust statistics studies procedures whose behavior remains controlled under outliers and deviations from an assumed model.
The minimax asymptotic-bias problem chooses an estimator to minimize its largest limiting bias over a specified neighborhood of a reference distribution.
The -trimmed mean deletes the smallest and largest observations and averages the rest. Its population functional is
At a differentiable distribution symmetric about zero, put . The influence function of the -trimmed mean is
An M-estimator minimizes an empirical objective or, when differentiable, solves an estimating equation .
On a compact parameter set, a continuous deterministic objective with a unique minimum has a positive separation gap outside each neighbourhood of its minimizer. Uniform convergence in probability of the random objectives bounds the deterministic objective difference at any attained random minimizer by twice the supremum discrepancy. This proves statistical consistency of each measurable minimizing selection.
Uniform convergence in probability of empirical criteria on a compact set proves consistency for minimizers in that set; it does not confine a minimizer over a larger domain. A bounded identifiable regression function can approach its true value again at infinity. For median regression, has unique zero at zero, but a small positive fitted sample median has a second solution . This solution diverges as , although uniform convergence on any fixed compact interval holds. Constraining the estimator, or proving suitable global separation and localization, repairs the argument.
A Catoni mean estimator replaces each scaled observation by a nondecreasing influence function whose exponential envelope is controlled by a quadratic. This yields sub-Gaussian deviation bounds assuming only a finite variance.
A B-robust estimator has bounded influence function, equivalently finite gross-error sensitivity, at the reference distribution.
Statistical contamination replaces part of an ideal distribution or sample by arbitrary observations.
The Kolmogorov -neighborhood of a cumulative distribution function consists of all distribution functions satisfying
Huber contamination is the epsilon-contamination model in which an arbitrary distribution contributes an unknown fraction of the observations.
A one-step estimator starts from a plug-in estimate and adds the empirical mean of an estimated influence function. Under suitable nuisance-estimation and remainder conditions, this correction removes the plug-in estimator's first-order bias and produces an asymptotically linear estimator.
Gross-error sensitivity is the supremum of the absolute influence function over contamination points.
At a distribution with positive density at its median , the median influence function is away from .
A breakdown point is the smallest contamination fraction capable of making an estimator arbitrarily unreliable.
The finite-sample replacement breakdown point is the largest replacement fraction under which an estimator remains bounded over arbitrary replacement values.
The Huber loss is quadratic near zero and linear beyond a fixed cutoff; its derivative is the clipped Huber score.
A Huber loss applied to a discrete gradient combines quadratic penalization of small slopes with linear penalization of large slopes. This interpolates between squared-gradient smoothing and total variation denoising, allowing smooth regions while retaining an edge-preserving large-gradient regime. The threshold should be chosen consistently with the signal's units and discretization. It can reduce staircasing in total variation denoising without guaranteeing its elimination.
The median-of-means estimator partitions a sample, computes each group mean, and returns the median of those means.
The Tukey median maximizes halfspace depth, the smallest probability mass or empirical fraction in any closed halfspace containing the candidate point.
Nonparametric statistics uses models whose effective dimension can grow with the sample size instead of fixing a finite-dimensional parametric family.
Independent additive noise multiplies characteristic functions: . When the noise characteristic function is nonzero, division recovers . A smoothing kernel limits the frequencies used, controlling the amplification of sampling error where is small.
Empirical likelihood places probability masses on observed or data-supported values and maximizes their product subject to normalization and any estimating constraints.
A nonparametric maximum-likelihood estimator maximizes a likelihood over a nonparametric family of distributions or functions rather than over a fixed finite-dimensional parameter space.
Density estimation constructs an estimate of an unknown probability density function from sampled observations.
Articles were limited to the first 100 out of 316 total.
Articles by others on the same topic
Statistical inference is a branch of statistics that involves drawing conclusions about a population based on a sample of data taken from that population. It provides the framework for estimating population parameters, testing hypotheses, and making predictions based on sample data. The primary goal of statistical inference is to infer properties about a population when it is impractical or impossible to collect data from every member of that population.