Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: split-plot experimental design

Donald D. Dorfman; Kevin S. Berbaum; Russell V. Lenth; Yeh Fong Chen

doi:10.1117/12.349629

24 May 1999 Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: split-plot experimental design

Donald D. Dorfman, Kevin S. Berbaum, Russell V. Lenth, Yeh Fong Chen

Author Affiliations +

Proceedings Volume 3663, Medical Imaging 1999: Image Perception and Performance; (1999) https://doi.org/10.1117/12.349629
Event: Medical Imaging '99, 1999, San Diego, CA, United States

Abstract

The major purpose of this paper was to evaluate the Dorfman/Berbaum/Metz (DBM) method for analyzing multireader receiver operating characteristic (ROC) discrete rating data on reader split-plot and case split-plot designs. It is not always appropriate or practical for readers to interpret imaging studies of the same patients in all modalities. In split plot designs, either a different sample of readers is assigned to each modality or a different sample of cases is assigned to each modality. For each type of split-plot design, a series of null-case Monte Carlo simulations were conducted. The results suggest that the DBM method provides trustworthy alpha levels with discrete ratings when ROC area is not too large, and case and reader sample sizes are not too small. In other situations, the test tends to be somewhat conservative. Our Monte Carlo simulations show that the DBM multireader method can be validly extended to the reader-split and case- split plot designs.

Citation Download Citation

Donald D. Dorfman, Kevin S. Berbaum, Russell V. Lenth, and Yeh Fong Chen "Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: split-plot experimental design", Proc. SPIE 3663, Medical Imaging 1999: Image Perception and Performance, (24 May 1999); https://doi.org/10.1117/12.349629

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available