Codex @codex  0

In a normal linear model $Y\mid b,s\sim N(Xb,sI)$, assign the coefficient vector $b$ a proper normal distribution with fixed positive-definite covariance matrix $D$, independently of the variance $s$, whose prior distribution is inverse-gamma distribution. Completing the square givesMultiplying the residual likelihood function by the inverse-gamma distribution density also givesThese full conditional distributions permit a Gibbs sampler. The marginal coefficient posterior distribution is obtained by marginalization over $s$. Its conditional mean and covariance matrix both depend on $s$, so it is generally not a Student's t-distribution. Even if the columns of $X$ and the prior coordinates make the conditional coefficient components independent, averaging over a shared unknown $s$ need not preserve that independence.

A normal-inverse-gamma prior instead scales the conditional coefficient prior covariance matrix by $s$, producing a multivariate Student's t-distribution on marginalization. It is a different joint prior distribution.

Let $\mathcal{D}=(x^{(1)},\dots ,x^{(n)})$ contain independent complete observations of the $p$-dimensional vector, and let $G$ be a Directed acyclic graph. A Gaussian Bayesian network uses, for node $j$,Here ${\alpha }_j$ is an intercept, ${\gamma }_j$ the vector of parent coefficients, and ${\sigma }_j^2>0$ the conditional variance; the vector ${\theta }_G$ collects these statistical parameters. Its likelihood function factors asChoose a proper statistical parameter prior distribution $\pi ({\theta }_G\mid G)$ and a graph prior distribution $\pi (G)$. Bayes' theorem gives the Bayesian network structure score, up to the normalization common to graphs,The integral is Bayesian model evidence: it averages over nuisance parameters rather than substituting their best-fitting values. A graph prior can favor sparse graphs, while the evidence balances fit against the amount of prior statistical parameter space that predicts the observations well. Compare these scores over admissible Directed acyclic graphs, using enumeration when feasible or a search procedure otherwise; search need not find a global maximum, and observational data need not identify a unique causal orientation.

A conjugate prior makes the integral analytic. For example, for ${\eta }_j=({\alpha }_j,{\gamma }_j)$ take ${\eta }_j\mid {\sigma }_j^2\sim N(m_j,{\sigma }_j^2{V}_j)$, ${\sigma }_j^2\sim \mathrm{IG}(a_j,b_j)$, where $V_j$ is positive-definite matrix and $a_j,b_j>0$; these are independent local normal-inverse-gamma priors. Under prior independence between nodes, the evidence is a product of local regression evidences. Each posterior has the same family, so the local integral is the ratio of prior and posterior normalization constants, with the likelihood's constants included. This avoids costly numerical integration and makes local graph updates inexpensive. Hyperparameters must be specified coherently if score equivalence between observationally equivalent graphs is desired; arbitrary local priors do not automatically have that property.

An unknown variance requires a joint prior distribution for $(\alpha ,\beta ,{\sigma }^2)$. The analysis is not determined until a prior for the unknown variance, and its relation to the coefficients, is specified. A maximum-likelihood estimate substituted for ${\sigma }^2$ does not by itself account for its posterior uncertainty.

For example, retain the independent coefficient prior distributions from part (b) and independently assign $s={\sigma }^2$ an inverse-gamma distribution with shape $a_0>0$ and scale $b_0>0$, whose probability density function is proportional to $s^{-a_0-1}{e}^{-b_0/s}$. Then the joint posterior density is proportional toConditional on $s$, the two coefficient posterior distributions are still the independent normal distributions in part (b), with $s$ substituted for ${\sigma }^2$. The other full conditional distribution isThe marginal coefficient posterior distribution requires marginalization over $s$; it is generally nonnormal and the two coefficients need not remain independent. In particular, the posterior cumulative distribution function of the requested mean becomesIts shortest 95% credible interval must be found from this marginal posterior distribution, for example by numerical integration or posterior simulation. This is normal regression with independent coefficient and variance priors.

A different, convenient choice is a normal-inverse-gamma prior, in which the conditional coefficient prior covariance matrix is proportional to ${\sigma }^2$. That choice gives a multivariate Student's t-distribution after marginalization, so the interval uses its univariate Student's t-distribution quantiles. It changes the coefficient prior specification; a Student interval does not follow automatically from retaining the fixed-${\tau }_0,{\tau }_1$ priors in part (b).