Human epidermal growth factor receptor 2 (HER2) serves as a prognostic and predictive biomarker for breast cancer. Recently, there has been an increasing number of studies evaluating the feasibility of utilizing H&E WSIs for determining HER2 status through innovative data-driven deep learning methods, taking advantage of the ubiquitous availability of H&E WSIs. One of the main challenges with these data-driven methods is the need for large-scale datasets with high quality annotations, which can be expensive to curate. Therefore, in this study, we explored both the region-of-interest (ROI)-based supervised and the attention-based multiple-instance-learning (MIL) weakly supervised methods for predicting HER2 status on H&E WSIs to evaluate whether avoiding labor-intensive tumor annotation will compromise the final prediction performance. The ROI-based method involved an Inception-v3 along with an aggregation step to combine the patch-level predictions into a WSI-level prediction. On the other hand, the attention-based MIL methods explored ImageNet pretrained ResNet, H&E image pretrained ResNet, and H&E image pretrained vision transformer (ViT) as encoders for WSI-level HER2 prediction. Experiments are carried out on N = 355 WSIs available in public domain with HER2 status determined by IHC and ISH and annotations of breast invasive carcinoma. The dataset was split into training/validation/test set with 80/10/10 ratio. Our results demonstrate that the attention-based ViT MIL method is able to reach similar accuracy as the ROI-based method on the independent test set (AUC of 0.79 (95% CI: 0.63-0.95) versus 0.88 (95% CI: 0.63-0.9) respectively), and thus reduces the burden of labor-intensive annotations. Furthermore, the attention mechanism enhances interpretability of the results and offers insights into the reliability of the predictions.
Tumor mutation burden (TMB) is an important biomarker for the prediction of response to anti-PD-1 immunotherapies. Studies have shown that higher level of TMB (TMB-H) is associated with higher response rate to immunotherapies in patients with various types of advanced solid tumors. However, the measurement of TMB depends on whole exome sequencing (WES) which is an expensive assay and not always available in standard clinical oncology settings. In this work, we assess the feasibility of predicting TMB-H based upon hematoxylin and eosin (H&E)-stained histopathology images, which is a routinely conducted assay in clinical oncology. Using an Inception-V3 convolutional neural network (CNN) as a baseline feature extractor, we compare adding a multi-layer perceptron (MLP) and a squeeze-and-excitation (SE) network on top of the baseline CNN. Training from random initialization and tuning with pretrained weights are also compared. Experiments are conducted on the H&E whole-slide images (WSI) of the melanoma dataset of The Cancer Genome Atlas (TCGA). Results from a 4-fold cross-validation show that the highest average area under the receiver operating characteristic curve (AUC) is 0.589, which implies that the prediction of TMB based on H&E WSI for melanoma remains a challenging problem that will warrant further investigations.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.