Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 207 / 2 / b

Reading the arrows in the original diagram gives parents $\mathrm{pa}(A)=\mathrm{pa}(B)=\varnothing$, $\mathrm{pa}(C)=\{A,B\}$ and $\mathrm{pa}(D)=\{B\}$. The Bayesian network factorization isFor binary variables, each conditional probability table has one free probability per parent configuration, since its two entries sum to one. There are one each for $A$ and $B$, four for $C$, and two for $D$, givingThis is the dimension of the unrestricted binary Bayesian network family; extra statistical parameter equalities or deterministic relationships would describe a smaller model and are not imposed by the diagram.

The original PDF shows the chain $A\to C\to B$, not a collider. Its Bayesian network factorization is $p(a,b,c)=p(a)p(c\mid a)p(b\mid c)$. For any $c$ with positive marginal probability,Integrating or summing over $a$ also gives the same conditional marginal $p(b\mid c)$. ThusThis is conditional independence; the graph does not generally imply marginal independence, since summing $p(a)p(c\mid a)p(b\mid c)$ over $c$ can transmit dependence from $A$ to $B$. Special statistical parameter choices can make marginal independence hold as well, so the claim is that only the conditional statement is guaranteed by the graph. Conditional distributions on null values of $C$ are immaterial to this assertion.

Let $\mathcal{D}=(x^{(1)},\dots ,x^{(n)})$ contain independent complete observations of the $p$-dimensional vector, and let $G$ be a Directed acyclic graph. A Gaussian Bayesian network uses, for node $j$,Here ${\alpha }_j$ is an intercept, ${\gamma }_j$ the vector of parent coefficients, and ${\sigma }_j^2>0$ the conditional variance; the vector ${\theta }_G$ collects these statistical parameters. Its likelihood function factors asChoose a proper statistical parameter prior distribution $\pi ({\theta }_G\mid G)$ and a graph prior distribution $\pi (G)$. Bayes' theorem gives the Bayesian network structure score, up to the normalization common to graphs,The integral is Bayesian model evidence: it averages over nuisance parameters rather than substituting their best-fitting values. A graph prior can favor sparse graphs, while the evidence balances fit against the amount of prior statistical parameter space that predicts the observations well. Compare these scores over admissible Directed acyclic graphs, using enumeration when feasible or a search procedure otherwise; search need not find a global maximum, and observational data need not identify a unique causal orientation.

A conjugate prior makes the integral analytic. For example, for ${\eta }_j=({\alpha }_j,{\gamma }_j)$ take ${\eta }_j\mid {\sigma }_j^2\sim N(m_j,{\sigma }_j^2{V}_j)$, ${\sigma }_j^2\sim \mathrm{IG}(a_j,b_j)$, where $V_j$ is positive-definite matrix and $a_j,b_j>0$; these are independent local normal-inverse-gamma priors. Under prior independence between nodes, the evidence is a product of local regression evidences. Each posterior has the same family, so the local integral is the ratio of prior and posterior normalization constants, with the likelihood's constants included. This avoids costly numerical integration and makes local graph updates inexpensive. Hyperparameters must be specified coherently if score equivalence between observationally equivalent graphs is desired; arbitrary local priors do not automatically have that property.