Past exam of the mathematics course of the University of Cambridge / 2004 / iii / Paper 38 / 5

Plot each observed mortality proportion vertically against its operation count horizontally, retaining the hospital labels. The required points, together with the target and approximate control bands used below, give the following hospital mortality funnel plot.

Hospitals sharing an operation count remain separate labeled points; equal observed mortality proportions at different volumes should not be treated as equally precise.

If $D\sim \mathrm{Bin}(n,p_0)$, the observed binomial proportion $\hat{p}=D/n$ has variance $p_0(1-p_0)/n$. Consequently its standard error is proportional to $n^{-1/2}$. A common target therefore produces broad uncertainty at low volume and narrower uncertainty at high volume, forming the funnel-shaped envelope around the horizontal target. The narrowing represents sampling precision, not a predicted change of mortality with volume.

Under the target model $p_0=0.20$, the normal approximation givesThe pointwise 95% binomial funnel control limits are thereforeDraw these curves around the horizontal 20% target, as in the figure. At $n=100$, the approximate limits are $0.1216$ and $0.2784$, so B and H are outside the nominal bands; that is a signal for investigation, not proof of inferior or superior care. The bands use the target variance rather than each hospital's observed rate. They are control limits for possible observations under the target model, not confidence intervals centered on the observed points.

Normal coverage is approximate, especially for the small hospitals; clipping does not repair that approximation. Exact binomial tail limits can be used if exact calibration is needed.

Four advantages of the hospital mortality funnel plot are:

It does not automatically adjust for patient severity, referral patterns or data quality, and an outlying point does not establish a causal explanation.

Let $I_j$ indicate a point outside its pointwise 95% limits. Under exact 95% marginal calibration, linearity givesIndependence is not needed for that expected count. If the eight hospital observations are independent, the probability of at least one false alarm isThus nominal pointwise 95% coverage is quite different from 95% simultaneous coverage of the whole display, a familywise error rate issue. Without independence, the product formula need not hold; the union bound is at most $8(0.05)=0.4$.

Because the permitted normal approximation gives only nominal 95% coverage for discrete binomial counts, these are nominal answers. With exact outside probabilities $q_j$, the exact expected count is ${\sum }_j{q}_j$ and, under independence, the exact probability is $1-{\prod }_j(1-q_j)$.

Three defensible approaches are:

A control-limit crossing prompts investigation; it is not itself a verdict on a hospital. Wider or adjusted thresholds also reduce sensitivity, so the false-alarm objective and the cost of missed genuine problems should be made explicit.