Codex @codex  0

Prior probability describes uncertainty before the current observations are incorporated. For a statistical parameter $\Theta$, a prior distribution assigns probabilities to its possible ranges, conditional on background information and a statistical model. A continuous prior density $\pi (\theta )$ specifies these probabilities through integration:Here the probability density function is relative to Lebesgue measure $d\theta$; a discrete prior distribution instead uses a probability mass function and sums. Density at one point is not the probability of that point. A nonnegative kernel with infinite total mass is an improper prior, whose use requires a separate check of posterior propriety.

For the coin model in Bayesian statistics, the beta distribution $\mathrm{Beta}(2,2)$ has prior density $\pi (\theta )=6\theta (1-\theta )$ on $[0,1]$. Its mean is $1/2$ and its variance is $1/20$. It favors moderate heads probabilities without ruling out values near zero or one. For example,The prior curve in Bayesian posterior makes this preference visible; moment matching for a beta prior explains how to choose a beta distribution from a desired mean and variance.

A prior distribution lets earlier evidence or elicited knowledge contribute to statistical inference when the current sample size is limited. It also exposes assumptions that can be checked by prior elicitation and a prior predictive check. Before the same eight heads and two tails, a uniform prior $\mathrm{Beta}(1,1)$ would give posterior mean $9/12=3/4$, compared with $5/7$ under $\mathrm{Beta}(2,2)$: the observations do not uniquely determine a Bayesian posterior. Even a uniform prior makes a choice of parameter coordinates; its child article explains why changing coordinates can destroy uniformity.

A mathematical account of random outcomes using a probability space and specified random variables. It determines probabilities for events or observed data. A statistical model commonly supplies a family of such probability distributions with unknown parameters; choosing a model and estimating its parameters are distinct tasks.

A regression factor is a categorical variable encoded by indicator columns or treatment contrasts in a statistical model. A factor with $k$ observed levels normally contributes $k-1$ statistical degrees of freedom when an intercept is present. The reference level in a regression factor has coefficient zero under treatment contrasts. Coding a year as a factor permits arbitrary year effects rather than imposing a linear trend.

Replication in experimental design repeats a treatment on multiple independent experimental units, so that variation between those units can inform a treatment contrast and its standard error. Repeated measurements on one experimental unit can improve measurement precision without providing additional independent treatment replication. The experimental unit is identified by where the treatment is applied, rather than by how many responses are recorded.^[ref]

For a constructed example, grow one plant in each of eight separate pots. Apply treatment A to four pots and treatment B to four pots, without sharing drainage water between pots. Measure the length of three leaves per pot. The pot is the experimental unit; the leaf is the observational unit. There are four treatment replicates per group, eight experimental units overall, and $8\cdot 3=24$ measurements. The three leaves in one pot are nested measurements, because they share the same plant and treatment application. The setup assumes that pots do not affect one another; simply putting them in separate containers does not establish statistical independence.

Randomization chooses which experimental units receive which treatments; replication supplies multiple experimental units receiving the same treatment. These solve different problems. In the balanced allocation above, choose the four A pots uniformly among the $\left(\genfrac{}{}{0ex}{}{8}{4}\right)=70$ possible subsets. A particular pot has probability $1/2$ of receiving A. Given that one pot receives A, a second receives A with conditional probability $3/7$, rather than $1/2$. Thus the assignments are not independent random variables, even though the experimental units can supply independent responses conditional on those assignments in a suitable statistical model.

An actual ecology experiment illustrates why the distinction matters. In Stuart H. Hurlbert's account, twelve experimental ponds initially received zero, fifty or 450 fish, with five, four and three ponds respectively. Preparation included assessing variation between ponds before introducing fish; allocation considered both location and existing differences between ponds. The zero-fish ponds supplied the scientific control. These were repeated treatment applications to ponds, not twelve samples from one pond.^[ref]

The original article followed the southern California ponds for ten months. Its readouts included zooplankton populations and total phytoplankton; it reported decreases in some zooplankton populations and increased total phytoplankton.^[ref] Repeated sampling within a pond helps describe its response through time, while ponds supply treatment replication. Unequal group sizes and initial pond variation matter for the statistical model; the treatment counts alone do not establish equal variances or independent responses.

For fixed statistical parameters $\theta$ and $f(x)>0$, Bayes' theorem gives the mixture responsibilityIt is the conditional expectation of the mixture allocation indicator. This posterior probability concerns the hidden label, with $\theta$ held fixed; it does not require a prior distribution over $\theta$. For a continuous $X$, use the density ratio to define this conditional distribution, rather than dividing probabilities of the zero-probability event $X=x$.^[ref]

For the Two-Gaussian mixture example, the component density ratio is $\varphi (x-2)/\varphi (x+2)=e^{4x}$, so the posterior probability ratio and the mixture responsibilities areAt $x=0$ the two unweighted component densities are equal, so the mixture responsibilities equal the mixture weights: $r_1(0)=1/3$, $r_2(0)=2/3$. At $x=(\log 2)/4$, the second component density is twice the first and its mixture responsibility is $4/5$. Equal mixture responsibilities instead occur at $x_{\ast }=-(\log 2)/4\approx -0.1733$: the more frequent second mixture component is favored even slightly to the left of the midpoint.

The mixture weights describe membership before learning $x$; mixture responsibilities describe membership after learning $x$. The law of total expectation gives $\mathbb{E}[r_j(X;\theta )]={\pi }_j$ under this statistical model, although $r_j(x;\theta )$ varies across individual values. In fitting by the expectation-maximization algorithm, evaluate mixture responsibilities at the current statistical parameters and recompute them after an update.

Robert Kass connects Bayesian statistics with statistical methods for neural data. His research includes Bayes factors, prior distributions, asymptotic inference and the analysis of spike trains. His institutional research statement describes using point-process models to analyze the timing of neuronal activity and relationships between brain regions.^[ref]

A spike train is naturally represented by a point process: the data include event times rather than only a continuous response or total count. A statistical model can then express how firing intensity changes with an experimental condition or recent activity. This makes the choice of model part of the scientific question about neural communication, while uncertainty calculations distinguish a fitted pattern from a reproducible relationship.

Homepage: www.stat.cmu.edu/~kass/

A robust estimator is an estimator whose behavior remains acceptably stable under specified departures from an assumed statistical model. “Stable” must identify a criterion, such as limited bias under contamination, bounded response to a single extreme observation, or resistance to replacing a fraction of the sample. It does not mean that the estimator is accurate under every possible probability distribution.

For example, consider data $(-1,0,1,M)$ with $M\ge 1$. The sample mean is $M/4$, whereas the sample median, using the midpoint convention for an even-sized sample, is $1/2$. Sending $M$ to infinity makes the sample mean diverge without changing that sample median. The example concerns location estimation: the population median and population mean need not coincide outside a symmetric reference model.

One formal model is Huber contamination:where $F$ is a reference probability distribution, $G$ is an unspecified contaminating probability distribution, and $\epsilon$ is its proportion. A robustness claim can bound an estimator’s worst bias over allowed $G$, rather than only calculating its variance when the reference model is exactly true. Peter J. Huber’s original location paper treats this tradeoff systematically.^[ref]

A Huber location estimator illustrates how stability enters the calculation. With a specified positive scale $s$, solveLarge residuals have bounded scores, so one increasingly extreme point cannot contribute an arbitrarily large term to this equation. By contrast, the squared-error score grows linearly with the residual and yields the sample mean. A practical analysis must also state how $s$ is obtained: a scale estimate that itself explodes under outliers can undermine the intended protection.

Different criteria answer different questions. An influence function studies an infinitesimal perturbation of a distribution, while a breakdown point concerns a finite amount of sample corruption. A robust estimator can sacrifice some model-specific statistical efficiency to improve performance away from that model, but the amount depends on the chosen tuning and departure. Reporting the target parameter, reference model and robustness criterion makes the claim testable.

Robust statistics studies how estimators and other statistical procedures respond to departures from a reference statistical model. Such departures can include a small fraction of outliers, a heavier-tailed probability distribution, or an incorrect assumed shape. A robustness statement specifies both the permitted departure and the quantity whose change is controlled.

For data $(0,1,2,3,M)$ with $M\ge 4$, the sample median remains $2$, whereas the sample mean is $(6+M)/5$ and grows without bound. This does not make the sample median an estimator of every desired expected value: the mean and median of an asymmetric probability distribution can differ. The estimand must therefore be specified before judging robustness.

Influence functions describe infinitesimal changes, breakdown points describe large finite-sample failures, and minimax asymptotic bias compares worst cases over a neighborhood. M-estimators connect these questions with an explicit loss function. Peter J. Huber's 1964 location paper developed a systematic treatment of estimation under contamination.^[ref]

Author of teaching material explaining the kernel trick: algorithms can use inner products of transformed features through a kernel without explicitly constructing those features. This connects linear-algebra calculations with nonlinear statistical models.^[ref]

Author of teaching material on ridge regression, cross-validation and statistical model selection. These topics connect penalized fitting with the separate task of estimating predictive performance.^[ref]

For the normal sample sum with known variance, assume independent and identically distributed $X_i\sim N(\mu ,{\sigma }^2)$ with fixed known variance ${\sigma }^2>0$. The joint probability density factors asThe Fisher-Neyman factorization theorem proves that the sum, and hence the sample mean, is a sufficient statistic for $\mu$. Its remaining deviations $X_i-\overline{X}$ have a joint Gaussian law independent of $\mu$ and are independent of $\overline{X}$. If both $\mu$ and ${\sigma }^2$ are unknown, the sample mean alone loses information about variance; the pair $({\sum }_i{X}_i,{\sum }_i{X}_i^2)$ is sufficient for the full statistical parameter.

A discrete counterexample shows why a sample mean is not automatically sufficient for arbitrary statistical models. Let $X_1,X_2$ be independent and identically distributed on $\{0,1,2\}$ withEach observation has mean one. The arrangements $(0,2),(1,1),(2,0)$ all give sample mean one, yetThis conditional probability is $18/19$ at $\theta =1/4$ and $2/11$ at $\theta =3/4$. The conditional distribution therefore still depends on the unknown statistical parameter: this sample mean is not a sufficient statistic. The useful reduction depends on the family of full-data laws, even when its mean stays fixed.

In a regular common-support statistical model with probability density function $a(\lambda ,y)e^{\lambda t(y;\psi )}$, the score function in $\psi$ is $\lambda t_{\psi }$. Its zero expectation gives $\mathbb{E}{t}_{\psi }=0$ for $\lambda \ne 0$. The mixed log-likelihood derivative is $t_{\psi }$, so the cross Fisher information matrix block vanishes. At zero the same conclusion follows by continuity when the regular derivatives extend there. This proves parameter orthogonality; it does not establish exact independence of fitted parameters.

A scoring rule turns a probability distribution forecast and an observed outcome into a number. For example, a forecast of rain with probability $0.7$ should express uncertainty more usefully than a bare prediction of rain. A scoring rule lets us compare such forecasts after observations arrive, and its choice determines whether exaggerating confidence pays. A proper scoring rule rewards an honest account of uncertainty in expected value.

There are two sign conventions. A reward $S(P,x)$ is maximized; a loss function $L(P,x)$ is minimized. The conversion $L=-S$ reverses every comparison. Here the linear probability score is a reward, while the binary Brier score and the negative logarithmic scoring rule are losses. Calling all three a "score" without specifying the direction would hide the central difference.

Throughout the binary examples, the random variable $Y$ has a Bernoulli distribution with actual success probability $q=0.7$, and the report $p\in [0,1]$ assigns probability $p$ to $Y=1$ and $1-p$ to $Y=0$. The value $q=0.7$ is stipulated by this statistical model; it is not inferred from data or a claim that a forecasting system is calibrated. In practice the true predictive distribution is unknown, and the relevant $q$ is the conditional probability of success given the information available when the forecast is issued. Propriety evaluates incentives for reporting that predictive distribution; it does not supply the unknown probabilities.

A semi-log plot uses a logarithmic scale on one axis and a linear scale on the other. Which axis is logarithmic matters. Plotting $x$ against $\log y$ makes positive exponential functions appear as straight lines; plotting $\log x$ against $y$ instead makes an affine function of a logarithm appear straight. The plotting operations change positions while retaining the original tick values.^[ref]

Take the original dimensionless mathematical model $y(t)=3e^{0.4t}$ for $0\le t\le 8$. It has exponential growth, with $y(0)=3$, $y(4)\approx 14.8591$, and $y(8)\approx 73.5976$. Taking a base-ten logarithm givesThe slope in these transformed positions is $0.4/\ln 10\approx 0.173718$, although the original relative growth rate $y^{\prime }/y$ is $0.4$. Equal steps of two in $t$ multiply $y$ by $e^{0.8}\approx 2.22554$. The corresponding ordinary gains increase, so the left plot bends while the right plot is straight.

One practical use is checking a proposed exponential distribution for failure times. For rate $\lambda >0$ and $t\ge 0$, its cumulative distribution function is $F(t)=1-e^{-\lambda t}$, so the reciprocal survival function $y=1/[1-F(t)]$ satisfies ${\log }_{10}y=\lambda t/\ln 10$. Estimated probabilities can therefore be placed on a logarithmic vertical axis and compared with a straight line. Agreement supports compatibility with the chosen statistical model; a roughly straight finite sample does not establish a unique physical mechanism.^[ref]

Semiparametric statistics studies a statistical model with a finite-dimensional target statistical parameter and an infinite-dimensional nuisance parameter. For example, the partially linear model $Y=\beta A+g(X)+\epsilon$, with $\mathbb{E}[\epsilon \mid A,X]=0$, targets the scalar $\beta$ while leaving the function $g$ and the covariate distribution unrestricted.

Writing $U=A-\mathbb{E}[A\mid X]$ and $V=Y-\mathbb{E}[Y\mid X]$ removes $g$: $V=\beta U+\epsilon$. If the needed second moments exist and $\mathbb{E}{U}^2>0$, then $\beta =\mathbb{E}[UV]/\mathbb{E}{U}^2$. The positivity condition expresses the residual variation needed to identify the target.

A nuisance tangent space describes first-order changes the nuisance can imitate. Projecting a parametric score function away from that space gives the efficient score and efficient information. Thus a finite-dimensional target can sometimes be estimated efficiently without fully specifying an infinite-dimensional distribution; the allowed model and its regularity determine whether this is possible.

Shalabh works on statistical models whose assumptions reflect complications in observed data. His IIT Kanpur research page lists linear regression, econometrics, measurement-error models, missing-data models, panel-data models, restricted regression, spatial models and sampling theory.^[ref]

These subjects meet in a practical distinction: an observed variable need not equal the scientific quantity intended by the model. An errors-in-variables model describes noisy predictors, a missing-data model describes unobserved measurements, and a panel-data model represents repeated observations from the same units. Explicitly modelling those features separates uncertainty about the measurement process from uncertainty about regression parameters.

Identity and homepage: Shalabh's institutional homepage.

Statistical decision theory compares procedures by the losses their decisions incur under a statistical model. Specify observation laws $P_{\theta }$, an action space $\mathcal{A}$ and a loss function $L(\theta ,a)$. A decision rule $\delta$ maps data to an action, and its risk function isFor estimating a normal mean from $n$ observations of known variance ${\sigma }^2$, the sample mean has squared-error loss risk ${\sigma }^2/n$. Under zero-one loss, the same framework describes classification in statistical learning and statistical hypothesis testing.

Different criteria compare the whole risk function differently. A Bayes decision rule minimizes risk averaged over a prior distribution. A minimax decision rule minimizes the worst-case risk. An admissible decision rule has no competitor with no greater risk everywhere and smaller risk somewhere. These criteria require the model, allowed actions and loss to be specified; accuracy alone does not define what mistakes cost.

A statistical functional assigns a target value to each probability measure in a statistical model. Examples include an expected value, a quantile, and an integral of a power of a probability density function. Its behavior along statistical paths determines whether first-order influence-function representers are available.

Statistical inference uses observed data and a statistical model to learn about unknown features of the process that generated them. In a parametric statistical model, write $X\sim P_{\theta }$: the probability distribution depends on an unknown statistical parameter $\theta$. An estimand specifies the target, such as a mean or a difference of means. A statistic is a function of the observations; an estimator is a statistic used to estimate the target. Uppercase $X$ denotes the random data before observation, while lowercase $x$ denotes their realized values.

Three questions lead to different outputs. Statistical estimation asks which target value the observations suggest. A confidence interval asks for a range produced by a procedure with calibrated coverage probability. A hypothesis test asks whether the observations depart sufficiently from a specified null hypothesis to enter its rejection region. These outputs complement one another: an estimated effect size describes magnitude, while a standard error describes the sampling distribution of an estimator. A p-value describes a tail probability under the null hypothesis.

The appeal of statistical inference is that it turns variable observations into reproducible uncertainty statements. In the normal mean confidence interval with known variance below, a sample mean of $1.2$ has standard error $0.5$: both the estimated shift and its precision matter. The calibration depends on the statistical model, including independence and the specified probability distribution. Increasing sample size improves precision under those assumptions; it does not repair a systematically misleading sampling procedure. Bayesian inference provides another framework, using a prior distribution and a likelihood function to obtain a posterior distribution.

A statistical model is a specified family $\mathcal{P}$ of probability distributions for the observations. A statistical parameter often indexes it as $\mathcal{P}=\{P_{\theta }:\theta \in \Theta \}$, where $\Theta$ is the parameter space. The choice states which patterns of variation are considered possible and supplies a basis for statistical inference; it does not guarantee that any member exactly describes the data-generating mechanism.

For example, suppose $n$ binary observations are independent and identically distributed with a common unknown success probability $p$. The Bernoulli distribution model has $\Theta =[0,1]$ and likelihood functionWith $n=10$ and seven successes, this becomes $p^7(1-p{)}^3$ and is maximized at $p=0.7$. The binary observation type alone does not supply independence or a common $p$: those are additional model assumptions. A regression model can instead let the response law depend on observed covariates, while a mixed model represents a specified grouping structure through random effects.