Automated detection and aggressiveness classification of prostate cancer on Magnetic Resonance Imaging (MRI) can help standardize radiologist interpretations, and guide MRI-Ultrasound fusion biopsies. Existing automated methods rely on MRI features alone, disregarding histopathology image information. Histopathology images contain definitive information about the presence, extent, and aggressiveness of cancer. We present a two-step radiology-pathology fusion model, ArtHiFy, Artificial Histopathology-style Features for improving MRI-based prostate cancer detection, that leverages generative models in a multimodal co-learning strategy, enabling learning from resource-rich histopathology, but prediction using resource-poor MRI alone. In the first step, ArtHiFy generates artificial low-resolution histopathology-style features from MRI using a modified Geometry-consistent Generative Adversarial Network (GcGAN). The generated low-resolution histopathology-style features emphasize cancer regions as having less texture variations, mimicking densely packed nuclei in real histopathology images. In the second step, ArtHiFy uses these generated artificial histopathology-style features in addition to MR images in a convolutional neural network architecture to detect and localize aggressive and indolent prostate cancer on MRI. ArtHiFy does not require spatial alignment between MRI and histopathology images during training, and it does not require histopathology images at all during inference, making it clinically relevant for MRI-based prostate cancer diagnosis in new patients. We trained ArtHiFy using prostate cancer patients who underwent radical prostatectomy, and evaluated it on patients with and without prostate cancer. Our experiments showed that ArtHiFy improved prostate cancer detection performance over existing top performing prostate cancer detection models, with statistically significant differences.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.