WeLineation: crowdsourcing delineations for reliable ground truth estimation

Saksham Goel; Yash Sharma; Malte-Levin Jauer; Thomas M. Deserno

doi:10.1117/12.2551279

2 March 2020 WeLineation: crowdsourcing delineations for reliable ground truth estimation

Saksham Goel, Yash Sharma, Malte-Levin Jauer, Thomas M. Deserno

Proceedings Volume 11318, Medical Imaging 2020: Imaging Informatics for Healthcare, Research, and Applications; 113180C (2020) https://doi.org/10.1117/12.2551279
Event: SPIE Medical Imaging, 2020, Houston, Texas, United States

Abstract

Crowdsourcing is a concept to encourage humans all over the world to generate ground truth for classification data such as images. While frameworks for binary and multi-label classification exist, crowdsourcing of medical image segmentation is covered only by few work. In this paper, we present a web-based platform supporting scientists of various domains to obtain segmentations, which are close to ground-truth references. The system is composed of frontend, authentication, management, processing, and persistence layers which are implemented combining various javascript tools, the django web framework, an asynchronous celery task, and a PostgreSQL database, respectively. It is deployed on a kubernetes cluster. A set of image data accompanied by a task instruction can be uploaded. Users can be invited or subscribe to join in. After passing a guided tutorial of pre- segmented example images, segmentations can be obtained from non-expert users from all over the world. The Simultaneous Truth and Performance Level Estimation (STAPLE) algorithm generates estimated ground truth segmentation masks and evaluates the users performance continuously in the backend. As a proof of concept, a test-study with 75 photographs of human eyes was performed by 44 users. In just a few days, 2,060 segmentation masks with a total of 52,826 vertices along the mask contour have been collected.

Conference Presentation

Citation Download Citation

Saksham Goel, Yash Sharma, Malte-Levin Jauer, and Thomas M. Deserno "WeLineation: crowdsourcing delineations for reliable ground truth estimation", Proc. SPIE 11318, Medical Imaging 2020: Imaging Informatics for Healthcare, Research, and Applications, 113180C (2 March 2020); https://doi.org/10.1117/12.2551279

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available