Past exam of the mathematics course of the University of Cambridge / 2012 / iii / Paper 41 / 2

The two independent binomial distributions give fitted event probabilities $24/99$ and $12/100$. Their odds ratio and log odds ratio areUsing the log odds ratio variance from a two-by-two table,The stated normal approximation and quantile approximately two give the confidence intervalExponentiation gives the corresponding odds ratio interval $(1.0824,5.0876)$. Its lower endpoint exceeds one.

A fixed-effect meta-analysis models the studies as estimating a common underlying treatment effect. In a generic inverse-variance version, independent log odds ratios $y_i$ have approximate distributions $N(\delta ,v_i)$ and are pooled with weights $1/v_i$. The supplied forest plot instead has the Mantel–Haenszel pooled odds ratio weights: for event/nonevent cells $a_i,b_i,c_i,d_i$ and total $n_i$, the normalized weights are proportional to $b_i{c}_i/n_i$. They reproduce the printed percentages and yield a pooled odds ratio about $2.338$. This is still a common-effect analysis; the two weighting methods should not be silently identified.

A random-effects meta-analysis allows different true effects: $y_i\mid {\delta }_i\sim N({\delta }_i,v_i)$ with ${\delta }_i\sim N(\mu ,{\tau }^2)$. The between-study heterogeneity variance ${\tau }^2$ adds to sampling variance, giving marginal variance $v_i+{\tau }^2$. Its pooled estimate targets the mean of the study-effect distribution rather than one identical effect. Uncertainty about heterogeneity must also be considered.

The I-squared statistic of $6.6\%$ is a small estimate of excess variation on the heterogeneity scale. It does not mean that $6.6\%$ of studies are different, nor that this percentage of patients responds differently. The Cochran's Q statistic compares the observed variation with that expected from the studies' sampling errors; its approximate null distribution has five degrees of freedom here. A heterogeneity-test p-value of $0.374$ provides no rejection of the common-effect hypothesis.

The individual confidence intervals in the forest plot overlap substantially, although some point estimates are larger than others. Taken together, there is little evidence of between-study heterogeneity in these six studies. Neither the nonsignificant test nor the small I-squared statistic proves equal effects: with only six studies, detecting moderate heterogeneity can be difficult. A study being individually significant while another is not is also not itself a test of their difference.

The additional estimate points in the same direction as the pooled effect: its odds ratio is about $1.90$, compared with the original pooled value about $2.34$. Its standard error is relatively large, and the approximate two-standard-error interval for its log odds ratio is $0.64\pm 1.14=(-0.50,1.78)$, or about $(0.61,5.93)$ on the odds-ratio scale. This interval includes both no effect and the existing pooled effect. Thus it supplies compatible but imprecise evidence and would tend to pull the pooled estimate somewhat toward one while adding information. A change in the main conclusion is unlikely.

For scale, a conventional inverse-variance common-effect fit to the six rounded tabulated estimates can be updated by ${\hat{\delta }}_{\mathrm{new}}=(W\hat{\delta }+wy)/(W+w)$, with new weight $w=0.57^{-2}$. This gives the new study roughly eleven percent of the combined weight, a modest effect on the estimate. An exact update of the supplied Mantel–Haenszel pooled odds ratio needs the new two-by-two table, which the reported log ratio and standard error do not uniquely specify. The qualitative compatibility conclusion does not depend on pretending that the supplied plot used inverse-variance weights.

Use a subgroup analysis in meta-analysis comparing studies before and after 2000, with a direct test of the between-subgroup effect difference. Alternatively use meta-regression with year, or the before/after indicator, as a study-level predictor. Compare the effect estimates themselves: significant and nonsignificant results need not differ significantly.

Two reasons for caution are that the proposed contrast was selected after examining the results, creating a multiple testing issue, and that only six studies supply an observational comparison of study characteristics. Year can be confounded with dose, participant populations, follow-up or design, so an apparent year effect need not be caused by time itself. The small number of studies also makes these comparisons imprecise. These would be exploratory findings rather than confirmation of an effect-modification hypothesis.