= Parameter estimation
{title2=$\hat\theta$}
<Parameter estimation> uses observations to infer the unknown <parameters> of a <mathematical model>. The result depends on both the observations and the model: one must specify which quantities are unknown, how observations relate to them, and how discrepancies are assessed. <Least squares> and <maximum likelihood estimation> provide two common criteria.<NIST and SEMATECH e-Handbook of Statistical Methods>[§4.1.4.1, Linear Least Squares Regression, unknown parameters and squared deviations]{ref}
For an explicit example, suppose $y_i=\theta+\varepsilon_i$, where the same unknown level $\theta$ applies to all observations. Minimizing
$$
L(\theta)=\sum_{i=1}^n(y_i-\theta)^2
$$
gives
$$
L'(\theta)=2\sum_i(\theta-y_i)=0
\quad\Longrightarrow\quad
\hat\theta=\frac1n\sum_i y_i.
$$
Since $L''(\theta)=2n>0$, this is the unique minimum for $n>0$. Observations $2,4,6$ give the <sample mean> $\hat\theta=4$, with residuals $-2,0,2$. Independent <normal distributions> with a common variance also make this estimate the <maximum likelihood estimation> solution for the level.
An estimate is not the same as a known exact value. Under the independent equal-variance model, its <standard error> is estimated by $s/\sqrt n$, where $s^2$ is the <sample variance>. For the three illustrative observations, $s^2=4$ and the estimated standard error is $2/\sqrt3$. Correlated errors or a common measurement offset require different accounting; collecting more identically biased readings does not remove the offset.
<Identifiability> is a separate prerequisite. If a model depends only on $a+b$, data of that form can estimate their sum but cannot uniquely separate $a$ from $b$. Pairs $(1,3)$ and $(2,2)$ give the same sum. More repeated observations of that same relationship reduce uncertainty about the sum without identifying both <parameters>. State what the observations can determine before reporting numerical estimates.
Back to article page
