Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 207 / 2

A Type I error is rejecting a true null hypothesis. Here it means declaring a statistical association between the biomarker and bowel cancer when, in the population and under the association model being tested, there is no statistical association. It is a false positive association finding, not merely a large estimation error. A test with level $0.05$ limits this probability to $0.05$ under the null hypothesis, subject to valid calibration; the p-value is not the probability that the null hypothesis is true.

Under the intended additional assumption of mutually independent, continuously calibrated null p-values, each test fails to reject with probability $0.95$. The familywise error rate is thereforeThere is a qualification to the printed use of “not correlated”: zero pairwise correlation coefficients do not generally imply mutual independence. A multivariate normal distribution with zero off-diagonal covariances does give independence, but an arbitrary biomarker distribution need not. The formula above requires independent rejection events, not just uncorrelated measurements.

The usual intuition for positively dependent tests is that they reject together, reducing the effective number of opportunities for a false positive. A rigorous sufficient assumption is positive association of random variables for the rejection indicators $I_1,\dots ,I_{10}$. Products of their non-rejection indicators are decreasing functions; association and induction giveThus under this stronger positive-dependence assumption the probability is no larger, and perfect dependence gives $0.05$ instead of $0.4013$. Strict inequality is not forced by every form of positive dependence.

Pairwise positive correlation coefficients alone, as printed, is insufficient. For a counterexample choose a vector of ten exchangeable random variables that are rejection indicators as follows: with probabilities $0.545,0.450,0.005$, respectively, reject none, reject one uniformly selected test, or reject all ten. Thenbut the familywise error rate is $0.455>1-0.95^{10}$. This can be realized with valid null p-values: conditional on the indicators, independently draw $p_i$ uniformly on $(0,0.05)$ if $I_i=1$ and on $(0.05,1)$ otherwise. Every $p_i$ is uniform on $(0,1)$, and $\mathrm{Cov}(p_i,p_j)=0.25\mathrm{Cov}(I_i,I_j)>0$. Taking biomarker statistics $X_i=1-p_i$ gives pairwise positively correlated null statistics with exactly these one-sided tests. Therefore the unconditional larger-or-smaller claim needs a specified dependence model.

The Bonferroni correction compares every p-value with $0.05/100000=5\times 10^{-7}$. Only the first two displayed values lie below that threshold; all undisplayed values are at least the tenth displayed value. HenceFor the Benjamini-Hochberg procedure at false discovery rate level $q=0.1$, the rank-$i$ threshold is $qi/100000=i\times 10^{-6}$. The first nine displayed ranks meet their thresholds, while rank ten does not: $1.1\times 10^{-5}>10^{-5}$. Consequently nine is the intended count if every later rank also fails its threshold. The rule is a step-up rule, however:It does not stop at the first failed comparison. The full rejection count is therefore not determined by ten order statistics. For example, completing the list with $99990$ values equal to $1$ gives $k=9$, whereas completing it with $99990$ values equal to $0.02$ gives $k=100000$, since $p_{(100000)}=0.02\le 0.1$. Both completions respect the printed ten smallest p-values. Thus at least nine are rejected; exactly nine requires an extra assumption about the undisplayed ranks. In the second completion even the tenth marker is rejected, despite failing its own rank threshold. The false discovery rate guarantee also requires the usual validity and independence or appropriate positive-dependence assumptions, unlike the arbitrary-dependence Bonferroni correction.

Reading the arrows in the original diagram gives parents $\mathrm{pa}(A)=\mathrm{pa}(B)=\varnothing$, $\mathrm{pa}(C)=\{A,B\}$ and $\mathrm{pa}(D)=\{B\}$. The Bayesian network factorization isFor binary variables, each conditional probability table has one free probability per parent configuration, since its two entries sum to one. There are one each for $A$ and $B$, four for $C$, and two for $D$, givingThis is the dimension of the unrestricted binary Bayesian network family; extra statistical parameter equalities or deterministic relationships would describe a smaller model and are not imposed by the diagram.

The original PDF shows the chain $A\to C\to B$, not a collider. Its Bayesian network factorization is $p(a,b,c)=p(a)p(c\mid a)p(b\mid c)$. For any $c$ with positive marginal probability,Integrating or summing over $a$ also gives the same conditional marginal $p(b\mid c)$. ThusThis is conditional independence; the graph does not generally imply marginal independence, since summing $p(a)p(c\mid a)p(b\mid c)$ over $c$ can transmit dependence from $A$ to $B$. Special statistical parameter choices can make marginal independence hold as well, so the claim is that only the conditional statement is guaranteed by the graph. Conditional distributions on null values of $C$ are immaterial to this assertion.

Let $\mathcal{D}=(x^{(1)},\dots ,x^{(n)})$ contain independent complete observations of the $p$-dimensional vector, and let $G$ be a Directed acyclic graph. A Gaussian Bayesian network uses, for node $j$,Here ${\alpha }_j$ is an intercept, ${\gamma }_j$ the vector of parent coefficients, and ${\sigma }_j^2>0$ the conditional variance; the vector ${\theta }_G$ collects these statistical parameters. Its likelihood function factors asChoose a proper statistical parameter prior distribution $\pi ({\theta }_G\mid G)$ and a graph prior distribution $\pi (G)$. Bayes' theorem gives the Bayesian network structure score, up to the normalization common to graphs,The integral is Bayesian model evidence: it averages over nuisance parameters rather than substituting their best-fitting values. A graph prior can favor sparse graphs, while the evidence balances fit against the amount of prior statistical parameter space that predicts the observations well. Compare these scores over admissible Directed acyclic graphs, using enumeration when feasible or a search procedure otherwise; search need not find a global maximum, and observational data need not identify a unique causal orientation.

A conjugate prior makes the integral analytic. For example, for ${\eta }_j=({\alpha }_j,{\gamma }_j)$ take ${\eta }_j\mid {\sigma }_j^2\sim N(m_j,{\sigma }_j^2{V}_j)$, ${\sigma }_j^2\sim \mathrm{IG}(a_j,b_j)$, where $V_j$ is positive-definite matrix and $a_j,b_j>0$; these are independent local normal-inverse-gamma priors. Under prior independence between nodes, the evidence is a product of local regression evidences. Each posterior has the same family, so the local integral is the ratio of prior and posterior normalization constants, with the likelihood's constants included. This avoids costly numerical integration and makes local graph updates inexpensive. Hyperparameters must be specified coherently if score equivalence between observationally equivalent graphs is desired; arbitrary local priors do not automatically have that property.