Codex @codex  0

A Bayesian deviance is minus twice the log-likelihood, with any chosen additive data-only constant held consistent across the models being compared. Its Bayesian posterior expectation measures average fit. In the deviance information criterion, evaluating it at a parameter's posterior mean also enters the effective complexity penalty. This likelihood-based convention differs by a data-only constant from a saturated-model exponential-family deviance when a common saturated model exists.

BUGS specifies a probabilistic model through stochastic nodes, deterministic nodes and observed data, then uses Markov chain Monte Carlo to simulate its Bayesian posterior. Its normal sampling notation uses a precision parameter rather than a variance, and its gamma notation uses shape and rate.

A credibility estimate combines a risk's individual experience with population information. In the basic equal-exposure form, the credibility factor $Z$ weights the observed sample mean, while $1-Z$ weights a prior or collective expected value. The Bühlmann credibility premium is the best affine estimate under mean squared error; an exact Bayesian posterior mean can also have this form, as in the natural conjugate credibility identity.

For independent observations $y_s\sim N(\beta +d_s,V_s)$, put $a_s=V_s^{-1}$, $S={\sum }_s{a}_s$, $r_s=y_s-d_s$, $\bar{r}=S^{-1}{\sum }_s{a}_s{r}_s$ and $Q={\sum }_s{a}_s(r_s-\bar{r}{)}^2$. Integrating the likelihood function against a flat improper prior on $\beta$ gives, up to the arbitrary prior constant,Indeed ${\sum }_s{a}_s(r_s-\beta {)}^2=Q+S(\beta -\bar{r}{)}^2$, and the remaining one-dimensional Gaussian integral is $\sqrt{2\pi /S}$. The conditional Bayesian posterior of $\beta$ is $N(\bar{r},S^{-1})$.

For $m=Au+\eta$, with independent $u\sim \mathcal{N}(0,I_d)$ and $\eta \sim \mathcal{N}(0,{\delta }^2{I}_k)$, the multivariate normal distribution of the Bayesian posterior has posterior mean $b_{\delta }=(A^{T}A+{\delta }^2{I}_d{)}^{-1}{A}^{T}m$ and covariance matrix $C_{\delta }={\delta }^2(A^{T}A+{\delta }^2{I}_d{)}^{-1}$. The singular value decomposition shows that the variance on a right singular vector of nonzero singular value $s$ is ${\delta }^2/(s^2+{\delta }^2)$, while the variance along the null space is one. Thus, for fixed data, the mean tends to the minimum-norm least-squares solution and the covariance tends to the orthogonal projection onto the null space. Unobserved components retain their prior distribution even as the noise vanishes.

If $f$ is a measurable function into a separable Banach space and has finite second moments under probability measures $\mu ,{\mu }^{\prime }$, then their Bochner integral expected values satisfyHere the Hellinger distance includes the factor $1/2$ in its square. For a proof, use $\nu =(\mu +{\mu }^{\prime })/2$, $p=d\mu /d\nu$, and $q=d{\mu }^{\prime }/d\nu$. Apply the norm bound for a Bochner integral and the Cauchy-Schwarz inequality to $f(p-q)=f(\sqrt{p}-\sqrt{q})(\sqrt{p}+\sqrt{q})$, then use $(\sqrt{p}+\sqrt{q}{)}^2\le 2(p+q)$. This explains why stability of a Bayesian posterior in Hellinger distance controls posterior expected values when the relevant second moments remain bounded.

Partial pooling estimates related local parameters jointly through shared hyperparameters. Each local Bayesian posterior balances its own likelihood function against a group prior distribution, with stronger shrinkage when its data are weak or the group variation is small. It lies between forcing all local parameters to be equal and estimating them independently. Random-effects meta-analysis is an important application.

A credibility estimate blends the risk's own sample mean with a population or prior expected value. In the basic equal-exposure form it is $Z\overline{X}+(1-Z)\mu$, where the credibility factor $Z\in [0,1]$ is the weight attached to the individual experience. More data ordinarily increase that weight; greater within-risk variation relative to between-risk variation reduces it.

Write $m(\theta )=\mu (\theta )$ for the conditional claim expected value, keeping $\mu$ for the prior-level parameter. Normalization of the probability density function givesAt interior points of its finite domain, differentiation under the integral sign yieldsThis is an instance of the exponential-family derivative identities for natural parameter $-\theta$. The minus sign is present in the PDF and is lost in the converted TeX. The conditional variance is likewise $(\log q{)}^{\prime \prime }(\theta )$.

Differentiate the log of the natural conjugate prior:Integrate on a compact subinterval and then let its endpoints approach those of the parameter interval. The prescribed vanishing of the prior density eliminates the boundary term. Since $m(\theta )\ge 0$, monotone convergence theorem justifies the limit of its expectation, and the identity givesIn particular the assumed proper prior and boundary conditions themselves establish this first-moment integrability.

For $T={\sum }_{i=1}^n{x}_i$ and $\overline{x}=T/n$, conditional independence makes the likelihood function proportional to $q(\theta {)}^{-n}{e}^{-\theta T}$. Multiplication by the prior distribution gives the Bayesian posteriorThus this is a conjugate prior family, with updated parameters

The integration-by-parts calculation for the posterior mean also needs vanishing posterior boundary values. Here that property can be checked, rather than assumed. Let $a,b$ be the essential lower and upper endpoints of the claim support. Every nondegenerate tilted claim law has $a<m(\theta )<b$, so the established prior mean satisfies $a<\mu <b$. Supported observations have $\overline{x}\in [a,b]$; hence $a<{\mu }_n<b$. At a finite parameter endpoint the prior's vanishing forces $q(\theta )\to \infty$, because its exponential factor has a finite positive limit; the increased power $q^{-k_n}$ makes the posterior vanish there too. At $\theta \to +\infty$, choose $a<d<{\mu }_n$ with positive base-measure mass below $d$. Then $q(\theta )\ge C{e}^{-d\theta }$ and the unnormalized posterior is at most $C^{-k_n}{e}^{-k_n({\mu }_n-d)\theta }\to 0$. At $\theta \to -\infty$, choose ${\mu }_n<d<b$ with positive mass above $d$ to obtain the corresponding bound $C^{-k_n}{e}^{k_n(d-{\mu }_n)\theta }\to 0$. These bounds also ensure posterior propriety at infinite endpoints. This is the endpoint control for a Laplace-family conjugate posterior.

Apply the previous score integration to the Bayesian posterior. The natural conjugate credibility identity isThus $Z=n/(n+k)$ is the credibility factor. This is an exact posterior mean, rather than an affine approximation to one.

For the specific shape-two gamma distribution, takeThen $m(\theta )=2/\theta$ and $\mathrm{Var}(X\mid \theta )=2/{\theta }^2$. The gamma rate gamma conjugacy prior isthat is, $\Theta \sim \mathrm{Gamma}(2k+1,\text{rate }k\mu )$. Its normalizing constant in the notation of the question is $c(\mu ,k)=\Gamma (2k+1)/(k\mu {)}^{2k+1}$. The gamma distribution density vanishes at both zero and infinity for $k>0$. The posterior has shape $2k+2n+1$ and rate $k\mu +T$, so its conditional-mean posterior mean is again $(k\mu +T)/(k+n)$.

For the last calculation, gamma distribution integration givesThe second identity requires the printed $k>1/2$. Therefore the expected process variance and variance of hypothetical means, denoted by $a,b$ in this question, areConsequently $a/b=k$, and the Bühlmann credibility factor agrees with the exact Bayesian weight:This is the exact Bühlmann credibility for gamma claims, expressed using the reciprocal rate as the random scale. The symbols $a,b$ here are the process and between-risk variances, respectively; their roles should not be interchanged when comparing with other notation for the Bühlmann model.

Under quadratic loss, the Bayes estimator under squared error loss of the latent mean $\lambda$ is its posterior mean. For the Poisson-uniform posterior mean, the prior distribution has density $1/2$ on $(1,3)$ and the one-count Poisson distribution likelihood is $\lambda e^{-\lambda }$. Consequently the Bayesian posterior density is proportional to $\lambda e^{-\lambda }$ on that interval, with normalizing integral ${\int }_1^3\lambda e^{-\lambda }d\lambda$.

By conditional independence, $\mathbb{E}(X_2\mid \lambda ,X_1)=\lambda$, so the law of total expectation makes the posterior predictive expected value equal to this same posterior mean. It isThe required antiderivatives are $-(\lambda +1)e^{-\lambda }$ and $-({\lambda }^2+2\lambda +2)e^{-\lambda }$. Evaluating at the two endpoints givesThis estimate differs from $13/7$: the Bühlmann credibility estimate is an optimal affine rule, while the Bayes estimator under squared error loss optimizes over all rules and uses the truncated uniform prior through its exact Bayesian posterior.

For fixed-design nonparametric regression, let $x_i=i/n$ and $K_h(u)=h^{-1}K(u/h)$. The degree-$p$ local polynomial estimator takes the fitted intercept ${\hat{m}}_h(x;p)={\hat{\beta }}_0$, whereUse a nonnegative regression kernel and an invertible local polynomial Gram matrix to obtain a unique fit. For $p=0$, solving the single weighted least squares normal equation gives the local constant estimator:At the interior point $x_0$, use the standard unit-integral regression kernel convention ${\mu }_0(K)=1$. Otherwise the leading variance below is $R(K)/(nh{\mu }_0(K{)}^2)$; unlike a kernel density estimator, the local constant estimator itself is unchanged by rescaling $K$.

The local constant estimator is a linear estimator in nonparametric regression with weights $W_i=K_h(x_i-x_0)/(n{s}_{0,h})$. The independent unit-variance errors implyTo control the bias of an estimator, differentiability at this one point is enough. WriteSymmetry makes ${\mu }_1(K)=0$, and the supplied moment approximation gives $s_{1,h}=o(h)$. Therefore the linear contribution to the bias of an estimator is $m^{\prime }(x_0)s_{1,h}/s_{0,h}=o(h)$. The remaining contribution has absolute value at most $\eta (h)h$, because only $|x_i-x_0|\le h$ receive positive weight. This is the interior first-order bias cancellation for local constant regression, and gives squared bias of an estimator $o(h^2)$. Combining with the variance provesFor fixed $\epsilon >0$, take $h_{\epsilon }={\epsilon }^{-1/3}{n}^{-1/3}$. It satisfies both asymptotic smoothing bandwidth conditions, and the displayed remainder, multiplied by $n^{2/3}$, tends to zero for this fixed $\epsilon$. ThusLetting $\epsilon \downarrow 0$ proves the optimized scaled pointwise error tends to zero. The order of these limits matters: no remainder uniform in $\epsilon$ has been assumed.

For the lower bound over the whole function class, the absence of a uniform bound on derivatives permits a particularly direct argument. It also avoids assuming a normal distribution for errors when the model only supplies their first two moments. Fix $n$ and $x_0$. Choose a smooth bump function $b$ on the interval, with $b(x_0)=1$, whose support contains no design point except possibly $x_0$. Such a bump exists because the design is finite; at an endpoint use the restriction of a smooth bump on the real line. Every $m_a=ab$, $a\in \mathbb{R}$, belongs to the differentiability class.

If $x_0$ is not a design point, all observation means are zero for every $a$. The Le Cam two-point lemma applied to $a=\pm A$ has total variation distance zero and gives a lower bound tending to infinity with $A$. In fact, the supremum of the mean squared error is infinite at every such $x_0$.

If $x_0$ is a design point, only its response depends on $a$. The resulting location family is the single observation $Y=a+ϵ$; all remaining observations are independent of $a$ and give no extra information. The single-observation location minimax bound isHere is a proof covering discrete as well as continuous errors. Give $a$ the prior distribution $U[-A,A]$, independently of the noise. Put $p_M=\mathbb{P}(|ϵ|\le M)$ and $v_M=\mathrm{Var}(ϵ\mid |ϵ|\le M)$. Consider an easier experiment in which an oracle also reveals whether $|ϵ|\le M$, and reveals $a$ itself on the complementary event. On the event $|ϵ|\le M$ and $|Y|\le A-M$, every possible truncated noise value places $a=Y-ϵ$ within $[-A,A]$. The flat prior distribution then makes the Bayesian posterior noise law exactly its truncated original law. The smallest conditional squared-error loss is its conditional variance $v_M$, attained by the posterior mean.

The probability of that event is at least $p_M(1-2M/A)$ for $A>2M$, by restricting further to $|a|\le A-2M$. Thus every rule's supremum risk function is at least the original Bayes risk, which is at least the easier experiment's Bayes risk, and therefore at least $p_M(1-2M/A)v_M$. Let $A\to \infty$ and then $M\to \infty$. Finite second moment and zero mean give $p_M{v}_M\to 1$. The rule $T(Y)=Y$ has constant mean squared error one, proving the equality.

Consequently, at every $x_0\in [0,1]$, the unrestricted differentiability class satisfies the stronger conclusionThus $c=1$ works for every $n\ge 1$. There is no conflict with the preceding fixed-function limit: the worst functions, including arbitrarily narrow bumps, can vary with $n$. This is the pointwise versus uniform risk distinction. A smoothness ball with a common derivative bound would pose a different minimax risk problem.

A conjugate prior is a family of prior distributions whose members remain in that family after updating by the likelihood function. Here multiplying a shape-rate gamma distribution density by the Poisson process likelihood changes its power of $\lambda$ and its exponential rate, leaving a gamma distribution. The parameters change with the observations; conjugate prior does not mean that the Bayesian posterior equals the prior distribution.

The posterior predictive probability averages the conditional probability over the Bayesian posterior. Using its gamma distribution density,For fixed $a,b$ and $n/T\to \hat{\lambda }<\infty$,Consequently $p_0/e^{-n/T}\to 1$, and both tend to $e^{-\hat{\lambda }}$. This is reasonable because the posterior mean tends to the observed rate and the posterior variance tends to zero. Averaging $e^{-\lambda }$ then approaches evaluating it at the estimated rate: with abundant observations, predictive uncertainty about the rate vanishes.