In this study, we trained a convolutional neural network (CNN) utilizing a mix of recent CNN architectural design strategies. Our goals are to leverage these modern techniques to improve the binary classification of kidney tumor images obtained using Multi-Photon Microscopy (MPM). We demonstrate that incorporating these newer model design elements, coupled with transfer learning, image standardization, and data augmentation, leads to significantly increased classification performance over previous results. Our best model averages over 90% sensitivity, specificity, accuracy, area under the receiver operating characteristic curve (AUROC) in image-level classification across cross-validation folds, superior to the previous best in all four metrics.
Convolutional neural networks (CNN) are a class of machine learning model that are especially well suited for imagebased tasks. In this study, we design and train a CNN on tissue samples imaged using Multi-Photon Microscopy (MPM) and show that the model can distinguish between chromophobe renal cell carcinoma (chRCC) and oncocytoma. We demonstrate the method to train a model using simple max-pooling vote fusion, and use the model to highlight regions of the input that cause a positive classification. The model can be tuned for higher sensitivity at the cost of specificity with a constant threshold and little impact to accuracy overall. Several numerical experiments were run to measure the model’s accuracy on both image and patient level analysis. Our models were designed with a dropout parameter that biases the model towards higher sensitivity or specificity. Our best performance model, as measured by area under the receiver operating characteristic curve (AUC of ROC, or AUROC) on patient level classification, is measured with a 94% AUROC and 88% accuracy, along with 100% sensitivity and 75% specificity.
A clear distinction between oncocytoma and chromophobe renal cell carcinoma (chRCC) is critically important for clinical management of patients. But it may often be difficult to distinguish the two entities based on hematoxylin and eosin (H and E) stained sections alone. In this study, second harmonic generation (SHG) signals which are very specific to collagen were used to image collagen fibril structure. We conduct a pilot study to develop a new diagnostic method based on the analysis of collagen associated with kidney tumors using convolutional neural networks (CNNs). CNNs comprise a type of machine learning process well-suited for drawing information out of images. This study examines a CNN model’s ability to differentiate between oncocytoma (benign), and chRCC (malignant) kidney tumor images acquired with second harmonic generation (SHG), which is very specific for collagen matrix. To the best of our knowledge, this is the first study that attempts to distinguish the two entities based on their collagen structure. The model developed from this study demonstrated an overall classification accuracy of 68.7% with a specificity of 66.3% and sensitivity of 74.6%. While these results reflect an ability to classify the kidney tumors better than chance, further studies will be carried out to (a) better realize the tumor classification potential of this method with a larger sample size and (b) combining SHG with two-photon excited intrinsic fluorescence signal to achieve better classification.
Distinguishing chromophobe renal cell carcinoma (chRCC) from oncocytoma on hematoxylin and eosin images may be difficult and require time-consuming ancillary procedures. Multiphoton microscopy (MPM), an optical imaging modality, was used to rapidly generate sub-cellular histological resolution images from formalin-fixed unstained tissue sections from chRCC and oncocytoma.Tissues were excited using 780nm wavelength and emission signals (including second harmonic generation and autofluorescence) were collected in different channels between 390 nm and 650 nm. Granular structure in the cell cytoplasm was observed in both chRCC and oncocytoma. Quantitative morphometric analysis was conducted to distinguish chRCC and oncocytoma. To perform the analysis, cytoplasm and granules in tumor cells were segmented from the images. Their area and fluorescence intensity were found in different channels. Multiple features were measured to quantify the morphological and fluorescence properties. Linear support vector machine (SVM) was used for classification. Re-substitution validation, cross validation and receiver operating characteristic (ROC) curve were implemented to evaluate the efficacy of the SVM classifier. A wrapper feature algorithm was used to select the optimal features which provided the best predictive performance in separating the two tissue types (classes). Statistical measures such as sensitivity, specificity, accuracy and area under curve (AUC) of ROC were calculated to evaluate the efficacy of the classification. Over 80% accuracy was achieved as the predictive performance. This method, if validated on a larger and more diverse sample set, may serve as an automated rapid diagnostic tool to differentiate between chRCC and oncocytoma. An advantage of such automated methods are that they are free from investigator bias and variability.
Background: Routine urological surgery frequently requires rapid on-site histopathological tissue evaluation either
during biopsy or intra-operative procedure. However, resected tissue needs to undergo processing, which is not only time
consuming but may also create artifacts hindering real-time tissue assessment. Likewise, pathologist often relies on
several ancillary methods, in addition to H&E to arrive at a definitive diagnosis. Although, helpful these techniques are
tedious and time consuming and often show overlapping results. Therefore, there is a need for an imaging tool that can
rapidly assess tissue in real-time at cellular level. Multiphoton microscopy (MPM) is one such technique that can
generate histology-quality images from fresh and fixed tissue solely based on their intrinsic autofluorescence emission,
without the need for tissue processing or staining.
Design: Fresh tissue sections (neoplastic and non-neoplastic) from biopsy and surgical specimens of bladder and kidney
were obtained. Unstained deparaffinized slides from biopsy of medical kidney disease and oncocytic renal neoplasms
were also obtained. MPM images were acquired using with an Olympus FluoView FV1000MPE system. After imaging,
fresh tissues were submitted for routine histopathology.
Results: Based on the architectural and cellular details of the tissue, MPM could characterize normal components of
bladder and kidney. Neoplastic tissue could be differentiated from non-neoplastic tissue and could be further classified as
per histopathological convention. Some of the tumors had unique MPM signatures not otherwise seen on H&E sections.
Various subtypes of glomerular lesions were identified as well as renal oncocytic neoplasms were differentiated on
unstained deparaffinized slides.
Conclusions: We envision MPM to become an integral part of regular diagnostic workflow for rapid assessment of
tissue. MPM can be used to evaluate the adequacy of biopsies and triage tissues for ancillary studies. It can also be used
as an adjunct to frozen section analysis for intra-operative margin assessment. Further, it can play an important role for
pathologist for guiding specimen grossing, selecting tissue for tumor banking and as a rapid ancillary diagnostic tool.
In this study, we propose a non-invasive method to distinguish pancreatic islet cells from exocrine cell clusters using multiphoton (MP) imaging. We demonstrate the principle of distinguishing them based on autofluorescence. The results show that MP imaging has a potential to distinguish pancreatic islets from exocrine cells. This ability to distinguish the two cell types could have many applications, such as the examination of fresh pancreatic biopsies when staining is not possible or desirable.
In clinical practice, histopathological analysis of biopsied tissue is the main method for bladder cancer diagnosis and
prognosis. The diagnosis is performed by a pathologist based on the morphological features in the image of a
hematoxylin and eosin (HE) stained tissue sample. This manuscript proposes algorithms to perform morphometric
analysis on the HE images, quantify the features in the images, and discriminate bladder cancers with different grades,
i.e. high grade and low grade. The nuclei are separated from the background and other types of cells such as red blood
cells (RBCs) and immune cells using manual outlining, color deconvolution and image segmentation. A mask of nuclei
is generated for each image for quantitative morphometric analysis. The features of the nuclei in the mask image
including size, shape, orientation, and their spatial distributions are measured. To quantify local clustering and alignment
of nuclei, we propose a 1-nearest-neighbor (1-NN) algorithm which measures nearest neighbor distance and nearest
neighbor parallelism. The global distributions of the features are measured using statistics of the proposed parameters. A
linear support vector machine (SVM) algorithm is used to classify the high grade and low grade bladder cancers. The
results show using a particular group of nuclei such as large ones, and combining multiple parameters can achieve better
discrimination. This study shows the proposed approach can potentially help expedite pathological diagnosis by triaging
potentially suspicious biopsies.
At the time of diagnosis, approximately 75% of bladder cancers are non-muscle invasive. Appropriate diagnosis and
surgical resection at this stage improves prognosis dramatically. However, these lesions, being small and/or flat, are
often missed by conventional white-light cystoscopes. Furthermore, it is difficult to assess the surgical margin for
negativity using conventional cystoscopes. Resultantly, the recurrence rates in patients with early bladder cancer are very
high. This is currently addressed by repeat cystoscopies and biopsies, which can last throughout the life of a patient,
increasing cost and patient morbidity. Multiphoton endoscopes offer a potential solution, allowing real time, noninvasive
biopsies of the human bladder, as well as an up-close assessment of the resection margin. While miniaturization
of the Multiphoton microscope into an endoscopic format is currently in progress, we present results here indicating that
Multiphoton imaging (using a bench-top Multiphoton microscope) can indeed identify cancers in fresh, unfixed human
bladder biopsies. Multiphoton images are acquired in two channels: (1) broadband autofluorescence from cells, and (2)
second harmonic generation (SHG), mostly by tissue collagen. These images are then compared with gold standard
hematoxylin/eosin (H&E) stained histopathology slides from the same specimen. Based on a "training set" and a very
small "blinded set" of samples, we have found excellent correlation between the Multiphoton and histopathological
diagnoses. A larger blinded analysis by two independent uropathologists is currently in progress. We expect that the
conclusion of this phase will provide us with diagnostic accuracy estimates, as well as the degree of inter-observer
heterogeneity.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.