Bayes estimator

A Bayes estimator minimizes posterior expected loss.

Under loss $w(\theta )|\theta -a|$, assume $0<\mathbb{E}[w(\theta )\mid x]<\infty$. The posterior expected loss is a positive constant times ordinary absolute-error loss under the probability density proportional to $w(\theta )\pi (\theta \mid x)$. Its minimizers are the medians of that weighted posterior: slopes of the loss change sign where the weighted probability below $a$ crosses one half. In particular, relative absolute loss has $w(\theta )=1/\theta$ for positive parameters. For a Gamma distribution posterior of shape two and rate $b$, the weighted posterior is an exponential distribution of rate $b$, so the estimate is $(\log 2)/b$.

A Bayes decision rule maximizes posterior expected utility, equivalently minimizes posterior expected loss, separately at each observed data value.

A decision rule $\delta$ is extended Bayes if, for every $\epsilon >0$, there is a proper prior $\pi$ with $r(\pi ,\delta )\le r^{\ast }(\pi )+\epsilon$, where $r^{\ast }(\pi )$ is the infimum Bayes risk. Equivalently a sequence of priors makes the rule's excess Bayes risk tend to zero. If its risk function is also the finite constant $c$, then $r^{\ast }(\pi )\ge c-\epsilon$, so every rule has maximum risk at least $c-\epsilon$. Letting $\epsilon \to 0$ proves that this equalizer rule is minimax.

Under squared error loss, the Bayes estimator is the posterior mean.

For independent $N(0,v)$ observations, the quadratic-loss risk of the displayed estimator is $v^2[n(n+2){\alpha }^2-2n\alpha +1]$. Within homogeneous rules its uniform minimum occurs at $\alpha =1/(n+2)$; within affine rules a nonzero optimal intercept improves the integrated risk of a homogeneous rule unless $\alpha =1/n$, when the prior moments are finite. More generally a proper prior cannot produce an exactly homogeneous posterior mean: writing its marginal kernel $m(s)$ for $s=|X{|}^2$, the identity $E[v\mid s]=\alpha s$ forces $m(s)=C{s}^{-1-1/(2\alpha )}$, whose induced density for $s$ is a pure power and is not integrable at both zero and infinity. Finite posterior quadratic losses, or finite Bayes risk for the integrated-risk argument, exclude vacuous infinite-loss minimization.

For $X\mid \theta \sim N_p(\theta ,I_p)$ and $\theta \sim N_p(0,c^2{I}_p)$, the posterior isThe Bayes estimator under quadratic loss is the posterior mean and its Bayes risk is the trace of the posterior covariance, $p{c}^2/(1+c^2)$.

For loss $L(\theta ,a)=q(\theta )(\theta -a{)}^2$ with $q(\theta )>0$, the Bayes estimator is the mean under the posterior reweighted by $q$:

Under $L(d,\theta )={\theta }^{-1}(\theta -d{)}^2$, the posterior expected loss is minimized atprovided the conditional inverse moment is finite and positive.

Posterior expected loss averages the loss over the posterior and is minimized pointwise in the data.