 Joann G. Elmore

Joann G. Elmore is a physician whose research examines variability in cancer diagnosis, including how consistently pathologists classify biopsy material. Her research group's description connects that work with statistics and machine learning for evaluating diagnostic decisions.^[ref]

In a 2017 primary study, Elmore and colleagues assembled 240 skin biopsy cases and gave participating pathologists sets of 36 or 48 cases. The design included a second interpretation of the same cases at least eight months later; 187 pathologists completed 8,976 interpretations in the first phase. Comparing an observer with their own later reading, with other observers and with a reference panel answers three different questions about statistical classification.^[ref]

The distinction is within-observer agreement versus reference agreement. In an illustrative set of 20 cases, an observer might repeat 18 of their earlier classifications but agree with a reference classification on only 14: the two proportions are $18/20=90\%$ and $14/20=70\%$. Repeating the same erroneous category increases the first proportion without improving the second. These are invented counts to explain the design, not results from Elmore's study. A consensus reference is an explicitly defined comparison standard; it does not make a panel infallible, and the diagnostic mix of a selected test set must be considered before interpreting an overall agreement percentage.