A Bayes estimator chooses an action minimizing posterior expected loss after observing data:The answer depends on the loss function: squared error gives the posterior mean, absolute error a posterior median, and discrete zero-one loss a posterior mode. For squared error, expanding proves the mean rule when the second moment is finite. Pointwise posterior minimization also minimizes integrated Bayes risk under the usual measurability and integrability conditions.
A posterior median is a median of the posterior distribution after observing data. Writing its cumulative distribution function as , a value is a posterior median when . This definition permits atoms and non-unique medians.
If the posterior has finite first absolute moment, its medians minimize posterior absolute-error loss . For a continuous posterior, differentiating with respect to gives ; the general case uses the corresponding left and right derivatives. Thus a minimizer is exactly a point with at least half the posterior mass on either side.
For a posterior placing probability at zero and at ten, the median is zero and its expected absolute loss is three. The posterior mean is three, whose absolute loss is . Choosing a Bayes estimator requires specifying the loss, not merely selecting a conventional summary of the posterior.
For loss , assume , and . The posterior expected loss is a positive constant times absolute-error loss under the posterior distribution reweighted by . Thus its minimizers are the medians of that weighted posterior distribution. More precisely, the weighted cumulative distribution function satisfies ; the one-sided slopes of the convex loss give this condition.
For relative absolute loss on positive parameters, . With a gamma distribution posterior of shape two and rate , reweighting produces an exponential distribution of rate , so the estimate is . The weighted first-moment condition ensures finite loss; without it every finite action can have infinite loss, making a median characterization uninformative.
A Bayes decision rule maximizes posterior expected utility, equivalently minimizes posterior expected loss, separately at each observed data value.
A decision rule is extended Bayes if, for every , there is a proper prior with , where is the infimum Bayes risk. Equivalently a sequence of priors makes the rule's excess Bayes risk tend to zero. If its risk function is also the finite constant , then , so every rule has maximum risk at least . Letting proves that this equalizer rule is minimax.
For independent observations, the quadratic-loss risk of the displayed estimator is . Within homogeneous rules its uniform minimum occurs at ; within affine rules a nonzero optimal intercept improves the integrated risk of a homogeneous rule unless , when the prior moments are finite. More generally a proper prior cannot produce an exactly homogeneous posterior mean: writing its marginal kernel for , the identity forces , whose induced density for is a pure power and is not integrable at both zero and infinity. Finite posterior quadratic losses, or finite Bayes risk for the integrated-risk argument, exclude vacuous infinite-loss minimization.
For and , the posterior isThe Bayes estimator under quadratic loss is the posterior mean and its Bayes risk is the trace of the posterior covariance, .
Under with , assume and . Reweighting the posterior distribution by and completing the square in gives the unique Bayes estimatorIndeed, the posterior expected loss is a constant plus . The moment conditions justify this expansion. For it is the posterior mean; parameter-dependent weighting generally changes the optimum.
For a positive parameter and loss , assume and . ThenThis quadratic has the unique minimizer , a harmonic rather than arithmetic posterior mean. Both moment conditions matter: finiteness of the inverse moment alone does not prevent the posterior expected loss from being infinite for every finite action.
Given observed data , the posterior expected loss of action iswhere is the posterior distribution and the loss function. Nonnegative losses define an extended nonnegative integral; meaningful comparisons require checking when finite loss is attainable. A Bayes decision rule chooses a measurable minimizer, or an approximate minimizer if an optimum is not attained.
With finite posterior second moment and squared-error loss, completing the square gives . Thus the posterior mean minimizes the loss. With finite first moment and absolute-error loss, a posterior median minimizes it instead. Posterior expected loss conditions on the observed data and averages over parameter uncertainty; the frequentist risk of a decision rule fixes the parameter and averages over possible data.
Articles by others on the same topic
There are currently no matching articles.