Codex @codex  0

For observations $\mathcal{D}$ and graph $G$, Bayes' theorem gives $p(G\mid \mathcal{D})\propto \pi (G)\int p(\mathcal{D}\mid {\theta }_G,G)\pi ({\theta }_G\mid G)d{\theta }_G$. Here $\pi (G)$ is the graph prior distribution and ${\theta }_G$ the local distribution statistical parameters. The integral is Bayesian model evidence; it averages over nuisance parameters with a proper statistical parameter prior distribution. For independent complete observations, the node-factorized likelihood function and independence of local statistical parameter prior distributions make the evidence factor over nodes; suitable conjugate priors can make the local integrals analytic. With missing node observations, integrating out unobserved values can couple the local parameters, so independence of local prior distributions alone does not guarantee this factorization. Proper priors and coherent hyperparameters matter for comparing different graphs.

For inverse-gamma distribution observations of known shape $k$ and scale $\Theta$, their likelihood function in $\Theta$ is proportional to ${\Theta }^{nk}\exp (-\Theta {\sum }_i{x}_i^{-1})$. A shape-$\alpha$, rate-$\lambda$ gamma distribution conjugate prior therefore gives the displayed Bayesian posterior. The Bayes estimator under squared error loss of the conditional claim expected value $\Theta /(k-1)$ is $(\alpha +nk)/[(k-1)(\lambda +{\sum }_i{x}_i^{-1})]$. It depends on reciprocals, rather than the sample mean of the claims.

A normal-gamma conjugate prior for the mean $\mu$ and precision parameter $\tau$ of a Gaussian distribution specifies a Gamma distribution $\tau \sim \mathrm{Gamma}(\alpha ,\beta )$ in the shape-rate convention and a conditional Gaussian distribution $\mu \mid \tau \sim N({\mu }_0,(K\tau {)}^{-1})$. Its joint probability density function is proportional to ${\tau }^{\alpha -1}\sqrt{\tau }\exp [-\beta \tau -K\tau (\mu -{\mu }_0{)}^2/2]$, with $\alpha ,\beta ,K>0$. For $n$ independent observations with Gaussian distribution $N(\mu ,{\tau }^{-1})$, multiplication by the likelihood and completing the square givesand ${\beta }_n=\beta +{\sum }_i(x_i-\bar{x}{)}^2/2+Kn(\bar{x}-{\mu }_0{)}^2/[2(K+n)]$. The identity behind conjugacy is $K(\mu -{\mu }_0{)}^2+{\sum }_i(x_i-\mu {)}^2=(K+n)(\mu -{\mu }_n{)}^2+{\sum }_i(x_i-\bar{x}{)}^2+Kn(\bar{x}-{\mu }_0{)}^2/(K+n)$.

Both the Metropolis–Hastings algorithm and the Gibbs sampler construct a Markov chain with a specified target distribution, usually a posterior distribution known only up to normalization. For a proposal density $q(y\mid x)$ and target density $\pi$, the Metropolis–Hastings acceptance probability isAfter drawing $y$, move there with this probability; otherwise stay at $x$. The unknown normalizing constant of $\pi$ cancels. The original Metropolis algorithm uses a symmetric proposal, $q(y\mid x)=q(x\mid y)$, so the proposal-density ratio disappears. The general Metropolis–Hastings algorithm permits nonsymmetric proposals and needs that correction.

A Gibbs sampler instead chooses a coordinate or block and draws directly from its full conditional distribution, leaving the other coordinates fixed. It is exactly a conditional Metropolis–Hastings update whose proposal is that full conditional. For a change from $x_i$ to $y_i$ with $x_{-i}$ fixed, the acceptance ratio isThus a Gibbs update is an always-accepted Metropolis–Hastings update with the full conditional as proposal. This statement concerns each update: a systematic sweep of several Gibbs kernels preserves the target but need not itself be reversible.

When the full conditional distributions are easy to sample, a Gibbs sampler avoids proposal tuning and rejection, and is particularly convenient for conjugate priors and data augmentation. When a conditional cannot be sampled directly, a Metropolis–Hastings algorithm can use only its evaluable density; such an update can also be inserted into a Gibbs sweep. Block or joint Metropolis–Hastings proposals can move efficiently along strongly correlated directions for which coordinate-wise Gibbs updates mix slowly. Conversely, a badly chosen Metropolis–Hastings proposal can reject most moves or make only tiny moves. Acceptance probability one by itself does not guarantee rapid exploration. Both methods require appropriate irreducibility and convergence checks before their long-run averages are useful posterior estimates.

Let $\bar{X}=n^{-1}{\sum }_i{X}_i$ and $S={\sum }_i(X_i-\bar{X}{)}^2$. The Gaussian distribution likelihood is proportional to ${\tau }^{n/2}\exp [-\tau {\sum }_i(X_i-\mu {)}^2/2]$. Multiplying by the normal-gamma distribution prior gives the quadratic exponent ${\sum }_i(X_i-\mu {)}^2+K_0(\mu -{\mu }_0{)}^2+2{\beta }_0$. Completing the square,The power of $\tau$ is ${\tau }^{{\alpha }_0+n/2-1}\sqrt{\tau }$, so the posterior distribution has exactly the required family withandFor a proper prior, $K_0,{\alpha }_0,{\beta }_0>0$ and ${\mu }_0\in \mathbb{R}$. The separate $\sqrt{\tau }$ factor is retained in both densities; it does not add $1/2$ to the update of $\alpha$. This is a conjugate prior for the Gaussian mean and precision parameter.

Put $s={\sum }_i{y}_i$. The independent-count likelihood function is proportional to $(1-\varphi {)}^n{\varphi }^s$. Its conjugate prior is $\boxed{\varphi \sim \mathrm{Beta}(a,b),a,b>0}$, giving the posterior distribution $\mathrm{Beta}(a+s,b+n)$.

A credibility estimate blends the risk's own sample mean with a population or prior expected value. In the basic equal-exposure form it is $Z\overline{X}+(1-Z)\mu$, where the credibility factor $Z\in [0,1]$ is the weight attached to the individual experience. More data ordinarily increase that weight; greater within-risk variation relative to between-risk variation reduces it.

Write $m(\theta )=\mu (\theta )$ for the conditional claim expected value, keeping $\mu$ for the prior-level parameter. Normalization of the probability density function givesAt interior points of its finite domain, differentiation under the integral sign yieldsThis is an instance of the exponential-family derivative identities for natural parameter $-\theta$. The minus sign is present in the PDF and is lost in the converted TeX. The conditional variance is likewise $(\log q{)}^{\prime \prime }(\theta )$.

Differentiate the log of the natural conjugate prior:Integrate on a compact subinterval and then let its endpoints approach those of the parameter interval. The prescribed vanishing of the prior density eliminates the boundary term. Since $m(\theta )\ge 0$, monotone convergence theorem justifies the limit of its expectation, and the identity givesIn particular the assumed proper prior and boundary conditions themselves establish this first-moment integrability.

For $T={\sum }_{i=1}^n{x}_i$ and $\overline{x}=T/n$, conditional independence makes the likelihood function proportional to $q(\theta {)}^{-n}{e}^{-\theta T}$. Multiplication by the prior distribution gives the Bayesian posteriorThus this is a conjugate prior family, with updated parameters

The integration-by-parts calculation for the posterior mean also needs vanishing posterior boundary values. Here that property can be checked, rather than assumed. Let $a,b$ be the essential lower and upper endpoints of the claim support. Every nondegenerate tilted claim law has $a<m(\theta )<b$, so the established prior mean satisfies $a<\mu <b$. Supported observations have $\overline{x}\in [a,b]$; hence $a<{\mu }_n<b$. At a finite parameter endpoint the prior's vanishing forces $q(\theta )\to \infty$, because its exponential factor has a finite positive limit; the increased power $q^{-k_n}$ makes the posterior vanish there too. At $\theta \to +\infty$, choose $a<d<{\mu }_n$ with positive base-measure mass below $d$. Then $q(\theta )\ge C{e}^{-d\theta }$ and the unnormalized posterior is at most $C^{-k_n}{e}^{-k_n({\mu }_n-d)\theta }\to 0$. At $\theta \to -\infty$, choose ${\mu }_n<d<b$ with positive mass above $d$ to obtain the corresponding bound $C^{-k_n}{e}^{k_n(d-{\mu }_n)\theta }\to 0$. These bounds also ensure posterior propriety at infinite endpoints. This is the endpoint control for a Laplace-family conjugate posterior.

Apply the previous score integration to the Bayesian posterior. The natural conjugate credibility identity isThus $Z=n/(n+k)$ is the credibility factor. This is an exact posterior mean, rather than an affine approximation to one.

For the specific shape-two gamma distribution, takeThen $m(\theta )=2/\theta$ and $\mathrm{Var}(X\mid \theta )=2/{\theta }^2$. The gamma rate gamma conjugacy prior isthat is, $\Theta \sim \mathrm{Gamma}(2k+1,\text{rate }k\mu )$. Its normalizing constant in the notation of the question is $c(\mu ,k)=\Gamma (2k+1)/(k\mu {)}^{2k+1}$. The gamma distribution density vanishes at both zero and infinity for $k>0$. The posterior has shape $2k+2n+1$ and rate $k\mu +T$, so its conditional-mean posterior mean is again $(k\mu +T)/(k+n)$.

For the last calculation, gamma distribution integration givesThe second identity requires the printed $k>1/2$. Therefore the expected process variance and variance of hypothetical means, denoted by $a,b$ in this question, areConsequently $a/b=k$, and the Bühlmann credibility factor agrees with the exact Bayesian weight:This is the exact Bühlmann credibility for gamma claims, expressed using the reciprocal rate as the random scale. The symbols $a,b$ here are the process and between-risk variances, respectively; their roles should not be interchanged when comparing with other notation for the Bühlmann model.

A conjugate prior is a family of prior distributions whose members remain in that family after updating by the likelihood function. Here multiplying a shape-rate gamma distribution density by the Poisson process likelihood changes its power of $\lambda$ and its exponential rate, leaving a gamma distribution. The parameters change with the observations; conjugate prior does not mean that the Bayesian posterior equals the prior distribution.

The specified gamma distribution uses shape $k$ and rate $\lambda$. Multiplying its density by the Poisson likelihood gives a posterior density proportional to ${\theta }^{k+S-1}{e}^{-(\lambda +n)\theta }$. Thus the conjugate prior calculation givesFor squared-error loss, the posterior expected loss isIt is minimized at the posterior mean, so the Bayes estimator isFor a fresh conditionally independent Poisson observation, the posterior predictive probability of zero is the posterior expectation of $e^{-\theta }$. Evaluating the gamma integral givesThe samples are independent conditional on $\theta$; integrating out the uncertain common parameter produces the predictive probability above.

Let $\mathcal{D}=(x^{(1)},\dots ,x^{(n)})$ contain independent complete observations of the $p$-dimensional vector, and let $G$ be a Directed acyclic graph. A Gaussian Bayesian network uses, for node $j$,Here ${\alpha }_j$ is an intercept, ${\gamma }_j$ the vector of parent coefficients, and ${\sigma }_j^2>0$ the conditional variance; the vector ${\theta }_G$ collects these statistical parameters. Its likelihood function factors asChoose a proper statistical parameter prior distribution $\pi ({\theta }_G\mid G)$ and a graph prior distribution $\pi (G)$. Bayes' theorem gives the Bayesian network structure score, up to the normalization common to graphs,The integral is Bayesian model evidence: it averages over nuisance parameters rather than substituting their best-fitting values. A graph prior can favor sparse graphs, while the evidence balances fit against the amount of prior statistical parameter space that predicts the observations well. Compare these scores over admissible Directed acyclic graphs, using enumeration when feasible or a search procedure otherwise; search need not find a global maximum, and observational data need not identify a unique causal orientation.

A conjugate prior makes the integral analytic. For example, for ${\eta }_j=({\alpha }_j,{\gamma }_j)$ take ${\eta }_j\mid {\sigma }_j^2\sim N(m_j,{\sigma }_j^2{V}_j)$, ${\sigma }_j^2\sim \mathrm{IG}(a_j,b_j)$, where $V_j$ is positive-definite matrix and $a_j,b_j>0$; these are independent local normal-inverse-gamma priors. Under prior independence between nodes, the evidence is a product of local regression evidences. Each posterior has the same family, so the local integral is the ratio of prior and posterior normalization constants, with the likelihood's constants included. This avoids costly numerical integration and makes local graph updates inexpensive. Hyperparameters must be specified coherently if score equivalence between observationally equivalent graphs is desired; arbitrary local priors do not automatically have that property.

For integer $\nu$, the sum ${\sum }_{j=1}^{\nu }{X}_j{X}_j^T$ of independent $N_p(0,V)$ vectors has a Wishart distribution. Its positive-definite density extends to real $\nu >p-1$ and is proportional to $|W{|}^{(\nu -p-1)/2}\exp [-\mathrm{tr}(V^{-1}W)/2]$. It is a conjugate prior for the precision matrix of a multivariate normal distribution. Its mean is $\nu V$.

The WinBUGS declaration dwish(R, nu) uses $R=V^{-1}$; the density convention is documented in the NIMBLE BUGS-language manual. For $\nu =p=3$ it is proper, while the expected inverse matrix does not exist: that expectation requires $\nu >p+1$. Specifying a Wishart prior on precision is therefore a substantive covariance assumption.