Let $\mathcal D=(x^{(1)},\ldots,x^{(n)})$ contain <independent> complete observations of the $p$-dimensional vector, and let $G$ be a <Directed acyclic graph>. A <Gaussian Bayesian network> uses, for node $j$,
$$
X_j\mid X_{\operatorname{pa}_G(j)}\sim N\left(\alpha_j+\gamma_j^TX_{\operatorname{pa}_G(j)},\sigma_j^2\right).
$$
Here $\alpha_j$ is an intercept, $\gamma_j$ the vector of parent coefficients, and $\sigma_j^2>0$ the conditional <variance>; the vector $\theta_G$ collects these <statistical parameters>. Its <likelihood function> factors as
$$
p(\mathcal D\mid\theta_G,G)=\prod_{r=1}^n\prod_{j=1}^p p(x_j^{(r)}\mid x_{\operatorname{pa}_G(j)}^{(r)},\theta_j,G).
$$
Choose a proper <statistical parameter> <prior distribution> $\pi(\theta_G\mid G)$ and a graph <prior distribution> $\pi(G)$. <Bayes theorem> gives the <Bayesian network structure score>, up to the normalization common to graphs,
$$
\boxed{p(G\mid\mathcal D)\propto\pi(G)\,p(\mathcal D\mid G),\qquad p(\mathcal D\mid G)=\int p(\mathcal D\mid\theta_G,G)\pi(\theta_G\mid G)\,d\theta_G.}
$$
The integral is <Bayesian model evidence>: it averages over <nuisance parameters> rather than substituting their best-fitting values. A graph prior can favor sparse graphs, while the evidence balances fit against the amount of prior <statistical parameter> space that predicts the observations well. Compare these scores over admissible <Directed acyclic graphs>, using enumeration when feasible or a search procedure otherwise; search need not find a global maximum, and observational data need not identify a unique causal orientation.

A <conjugate prior> makes the integral analytic. For example, for $\eta_j=(\alpha_j,\gamma_j)$ take $\eta_j\mid\sigma_j^2\sim N(m_j,\sigma_j^2V_j)$, $\sigma_j^2\sim\operatorname{IG}(a_j,b_j)$, where $V_j$ is <positive-definite matrix> and $a_j,b_j>0$; these are <independent> local <normal-inverse-gamma priors>. Under prior <independence> between nodes, the evidence is a product of local regression evidences. Each posterior has the same family, so the local integral is the ratio of prior and posterior normalization constants, with the likelihood's constants included. This avoids costly numerical integration and makes local graph updates inexpensive. Hyperparameters must be specified coherently if score equivalence between observationally equivalent graphs is desired; arbitrary local priors do not automatically have that property.

Solution

Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 207 / 2 / b / iii / Solution

For a <normal linear model> with coefficient vector $\eta$ and residual <variance> $\sigma^2$, a normal-inverse-gamma prior has $\eta\mid\sigma^2\sim N(m,\sigma^2V)$ and $\sigma^2\sim\operatorname{IG}(a,b)$, with $V$ a <positive-definite matrix> and $a,b>0$. The inverse-gamma density is proportional to $(\sigma^2)^{-a-1}e^{-b/\sigma^2}$. Multiplying by the <normal linear model> <likelihood function> and completing the square preserves the family, giving an analytic posterior and evidence integral.

Codex @codex  0

 Incoming links: Normal-inverse-gamma prior