Joann G. Elmore is a physician whose research examines variability in cancer diagnosis, including how consistently pathologists classify biopsy material. Her research group's description connects that work with statistics and machine learning for evaluating diagnostic decisions.[ref]
Joann G. Elmore research group at UCLA. “About Our Research Group”: cancer diagnosis, physician variability and statistical methods
In a 2017 primary study, Elmore and colleagues assembled 240 skin biopsy cases and gave participating pathologists sets of 36 or 48 cases. The design included a second interpretation of the same cases at least eight months later; 187 pathologists completed 8,976 interpretations in the first phase. Comparing an observer with their own later reading, with other observers and with a reference panel answers three different questions about statistical classification.[ref]
Pathologists' diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study by Joann G. Elmore and others. Abstract, Participants and Results; Methods, reference diagnoses; BMJ 357:j2813, DOI 10.1136/bmj.j2813
The distinction is within-observer agreement versus reference agreement. In an illustrative set of 20 cases, an observer might repeat 18 of their earlier classifications but agree with a reference classification on only 14: the two proportions are and . Repeating the same erroneous category increases the first proportion without improving the second. These are invented counts to explain the design, not results from Elmore's study. A consensus reference is an explicitly defined comparison standard; it does not make a panel infallible, and the diagnostic mix of a selected test set must be considered before interpreting an overall agreement percentage.
Articles by others on the same topic
There are currently no matching articles.