Funnel plot 2026-10-07
A funnel plot displays study effect estimates against a measure of their precision, often standard error increasing down the vertical axis. Under a common-effect model, imprecise estimates spread more widely around the effect. Asymmetry is evidence of a small-study effect and can be consistent with publication bias, but it has other possible explanations.
Four distinct objections, each with a corresponding improvement, are as follows.
  • Different patients imply different baseline risks. Raw death rates confound performance with case mix, including urgency, disease severity and comorbidity. Use a prespecified, validated risk-adjusted provider comparison, standardizing to comparable patients or comparing observed deaths with . Check overlap and model calibration rather than assuming adjustment removes all confounding.
  • Small annual volumes produce unstable rankings. A few deaths can move a surgeon far up or down a league table; zero deaths need not indicate zero risk. Display counts and suitably calibrated confidence intervals for binomial proportions, preferably in funnel plots against volume. Pool suitable periods or use a hierarchical Bayesian model to stabilize estimates, while allowing genuine time changes.
  • Many simultaneous comparisons generate chance outliers. With independent null comparisons, the probability of at least one false positive is , and ranks amplify selection of extremes. Use multiple testing adjustments or simultaneous control limits and seek confirmation before interpreting an apparent outlier. Dependence through the national comparator must also be accounted for.
  • The endpoint and unit of attribution can mislead. In-hospital death depends on discharge and transfer policy, and care is delivered by a team; follow-up after discharge is omitted. Use an independently audited fixed-horizon endpoint, such as a prespecified 30-day mortality outcome, linked across hospitals and to death registrations, and report an appropriate team or institutional comparison alongside any surgeon-level estimate. Reliable attribution and consistent inclusion criteria reduce incentives to manipulate coding or case selection.
A confidence interval containing the national rate does not demonstrate equivalence, and a raw ranking does not establish a causal performance difference. The improved presentation should convey precision and uncertainty as well as the estimated outcome.
A funnel plot places each estimated log risk ratio horizontally and its standard error vertically, with the most precise studies at the top. Under a common-effect model without selective availability, less precise estimates should spread approximately symmetrically around the underlying effect. The plotted dashed limits are the pooled estimate plus or minus twice the standard error; they illustrate the expected sampling spread, not a test of publication bias by themselves.
Figure 1.
Funnel plot of the ten transfusion trial estimates, with inverse-variance pooled effect and approximate sampling limits
. Details.
The least precise estimates are mostly far to the left, whereas the most precise estimates are close to zero. The reported regression slope means that increasing the standard error by one unit is associated with a decrease of about in the estimated log risk ratio. Its 95% confidence interval excludes zero, and gives evidence against a zero slope under that regression model. This is evidence of a small-study effect: less precise trials report stronger apparent benefit.
There is evidence consistent with publication bias, but asymmetry does not identify its cause. Selective availability of favourable small studies is a plausible explanation. Genuine differences in patient populations, trial quality, interventions or effect modification could also generate the pattern, and there are only ten trials. The given outcome-on-error regression should be interpreted as specified; it is not automatically the original standardized-effect-on-precision form of an Egger test. The plot and slope justify investigating missing studies and sensitivity to selection, rather than concluding that publication bias has been proved.
Publication bias 2026-10-07
Publication bias arises when the availability of studies depends on the results, such as their significance, magnitude or direction. A meta-analysis of the available studies can then differ systematically from one including all eligible evidence. Funnel plot asymmetry can motivate investigation but does not establish this mechanism; heterogeneity and other small-study effects can also cause asymmetry.
A risk-adjusted provider comparison contrasts observed outcomes with outcomes expected from a prespecified patient-risk model, or standardizes providers to a common patient population. For individual fitted risks , the expected event count is . Patient selection, model calibration, omitted risk factors, small samples and dependence between outcomes can still affect interpretation. Funnel plots display precision differences that an ordered league table can obscure.
Trim and fill 2026-10-07
Trim and fill estimates missing studies from funnel plot asymmetry by trimming extreme studies to locate a centre and filling in counterparts on the less represented side. The adjusted meta-analysis includes these imputed estimates. Its interpretation depends on a symmetry model for missing evidence; it is not a guarantee that genuine publication bias has been removed.