|
1.IntroductionSurgical resections are often the primary treatment option for both benign and malignant tumors. Curative surgery for many cancers requires a wide local excision to achieve negative margins.1,2 Although a greater extent of tumor resection is critical for reducing tumor recurrence and improving prognosis, wide negative margins for brain surgery are not possible.3–5 At the margins, the delicacy and unique functionality of the surrounding healthy tissues necessitate extreme precision during excision for neurosurgery, as the risk of damaging normal brain tissue is the impairment of patient’s cognitive function for their lifetime. In intraoperative neurosurgery, the ability to assess tumor margins with high accuracy would benefit patient’s survival, while preserving quality of life after surgery. Visual appearance and palpation have been the standard methods used by surgeons to differentiate tumor from healthy tissues. These techniques are ineffective at the margins for many tumors, where high contrast and resolution are required intraoperatively. Bright tumor-specific indicators, such as targeted fluorophore, can visually enhance tumor margins, which is ideal for residual tumor cleanup and removal of disturbed tumor fragments. Compared to other noninvasive imaging technologies, such as magnetic resonance imaging, computed tomography, positron emission tomography, and ultrasound, advanced optical imaging is the most pragmatic means to provide high-resolution, video-rate, and noncontact intraoperative imaging for surgeons. In the past 30 years, various optical imaging techniques have been investigated to guide surgery and to delineate tumor margins,6–12 which can be categorized into three mechanisms: (1) structural contrast enhancement, (2) metabolic contrast enhancement, and (3) molecular contrast enhancement.
To verify tumor margins detected by fluorescence images, the in vivo imaging must be compared with pathology, which provides the ground truth of cancer regions. The different tissue appearances from the two imaging modalities makes this multimodal registration task challenging. Moreover, variations in tissue optical properties and tumor depth make correlating fluorescence images to histopathological slides more difficult. Elliott et al.26 utilized IRDye800CW-labeled affibody specific to epidermal growth factor receptor to guide glioma resection. The tumor was correlated to fluorescence contrast using a multistep image registration. However, in their approach, fluorescence images were directly registered to histological slides. Structural landmarks in fluorescence images were selected as control points for registration, which are fewer and less obvious, compared to those in reflectance images. Lu et al.27 applied principal component analysis to extract macro images from HSI and registered these extracted macro images to histological images, thus evaluating HSI as an intraoperative visual aid for surgical resection. However, both registration methods compared to a single-layer histological slide, which was selected subjectively and might not contain complete tumor information. In this paper, a multistep registration framework is established to correlate fluorescence images to stacked histological images, which can be applied to evaluate fluorescent probes in delineating tumor cores and margins. Since variable experimental parameters and artifacts may affect accurate evaluation of fluorescence imaging, a series of preprocessing steps are applied to calibrate fluorescence imaging for improved accuracy and standardization, as stated in Ref. 28. An optical attenuation model is applied to integrate multiple histological slides and to project deeper tumor regions to the surface. Preliminary aspects of this model are presented in Image-Guided Procedures, Robotic Interventions, and Modeling panel within 2018 SPIE Medical Imaging.11 The whole framework is verified using a mouse model of brain tumor labeled with NIR molecular probe, BLZ-100 (Blaze Bioscience Inc., Seattle, Washington).24 Multimodal scanning fiber endoscope (mmSFE), a wide-field multimodal imaging device that captures fluorescence and reflectance simultaneously with exact spatial registration, is used to image BLZ-100 tumor indicator ex vivo. 2.Materials and Method2.1.Mouse Brain Tumor ModelA patient-derived orthotopic xenograft medulloblastoma tumor model (Med-114FH) was used for brain tumor fluorescence imaging under the IACUC approved protocol at the Fred Hutchinson Cancer Research Center (IR no. 1457).29 A total of 10 adult male and female NSG mice between 3 and 6 months of age were used in this study. Recipient mice were administered buprenorphine SR analgesia by subcutaneous injection before the orthotopic implant. At the time of surgery, mice were anesthetized using inhaled isoflurane. Establishment of anesthesia was determined by monitoring respiratory rate and toe pinch reflex. Under anesthesia, an incision of was made along the mediolateral line using a disposable scalpel and the skull was exposed and cleaned of any minor connective tissues or blood using cotton-tipped applicators. Once the area was clean, a handheld microdrill (Ideal Microdrill #67-1000, Cell Point Scientific) with a 0.9-mm burr (19007-09 Fine Scientific Tools) was used to create a burr hole into the cortex. About of cell suspension (100,000 cells) was then deposited into the intracranial space by inserting a Rainin’s Pipet-Lite fitted with a 2 to ART tips barrier nonfiltered pipette tip (2139, Thermo Scientific). A small piece of surgical foam (09-0396-05 Pfizer Injectables) was placed onto the burr hole and the incision was closed with veterinary-grade surgical glue (Vetbond, 3M). Following closure of the incision, mice were removed from isoflurane and transferred to a clean recovery cage placed on a heat mat for the duration of recovery. Mice were monitored regularly for signs of tumor formation, which can include distention of the calvarium, head tilt, reduced feeding, weight loss, dehydration, hunched posture, or poor grooming habits. Brain tumor symptoms were typically observed 4 to 6 weeks after implantation. 2.2.BLZ-100BLZ-100 is composed of a modified form of the CTX peptide conjugated to the NIR fluorescent dye ICG, which has been shown to bind specifically to many types of tumor cells, including glioma and other brain tumors, breast, head and neck cancers, and sarcomas.24,25,30 Assessment of tumor burden is made by trained researchers who use professional judgment and experience to determine that the mice have clinically evident tumor burden but are not moribund. On visual identification of the tumor burden, mice were administered 10 nmol of BLZ-100 in phosphate-buffered saline (PBS) by tail vein injection. A 30G needle attached to an insulin syringe was inserted into the right lateral tail vein of a mouse and the compound was administered as a single bolus injection. Mice were returned to the home cage with free movement and access to food and water for the 4-h incubation period. After 4 h of circulation, brains were harvested. The collected brains were stored in ice-cold PBS buffer, shielded from light and transferred for mmSFE imaging. 2.3.Multimodal Scanning Fiber Endoscope ImagingThe mmSFE imaging system is specifically designed for wide FOV, high-quality, video-rate (30 Hz), concurrent multiple fluorescence imaging.31,32 Three diode lasers at 488, 638, 785 nm are combined and sent to a single-mode optical fiber, which is driven by a piezo tube actuator and scanned in a spiral pattern [Fig. 1(a)]. Fluorescence and reflected light are then collected by a concentric ring of high numerical aperture multimodal optical fibers [Fig. 1(b)]. The mmSFE has an overall flexible shaft diameter of 1.2 to 2.4 mm and a rigid tip length of 9 mm [Fig. 1(c)], depending on the size and number of these collection fibers. Four imaging channels can be produced concurrently, one for reflectance (638 nm) and three for fluorescence including one NIR channel, using separate filters and Hamamatsu MPPC detectors. In this study, a 2.4-mm diameter endoscope was used for BLZ-100 imaging. The NIR fluorescence is projected to the green channel with the reflectance as grayscale. The whole brain surface was first imaged using the mmSFE at 1 cm away from the scope distal tip, which was then sectioned into 2-mm slices coronally. Tissue sectioning was performed using the sharpest instruments possible to allow for clean cutting and to prevent damage associated with crushing or tearing of tissue. Double-edged blades with a thickness of 0.10 mm (Electron Microscopy Sciences Cat. # 72003-01) were used to cut the brain that was 3 to 10 h post euthanasia. Cutting blocks were used for high-quality sectioning (Braintree Scientific 1 mm Coronal). Immediately after sectioning, the sliced brains were placed in ice-cold PBS buffer and imaged using mmSFE at 1 cm away from the scope tip. Reflectance and fluorescence images of the slices were captured with dimmed laboratory lights on. In summary, five slices were imaged using mmSFE for each mouse. 2.4.HistologyAfter mmSFE imaging, the 2-mm brain slices were fixed in 10% neutral buffered formalin solution for 48 h. Then, routine hematoxylin and eosin (H&E) staining was done for histological evaluation. The formalin-fixed tissues were embedded in paraffin, which were later cut into -thickness slides coronally at intervals. For each tissue block, 5 to 6 slides were cut. The cut slides were stained with H&E and imaged using a Leica color pathology slide scanner (Aperio Scanscope AT, Leica Biosystems Imaging, Inc.). The whole protocol for mmSFE imaging and histology processing is shown in Fig. 2. 2.5.Calibration of Near-Infrared Fluorescence Imaging for Tumor Margin Delineation2.5.1.Preprocessing for multimodal scanning fiber endoscope fluorescence imagingThe fluorescence data acquired using mmSFE were preprocessed for accurate evaluation and standardization, which consisted of the two steps shown below. Specular reflection detection and removalSpecular reflections are usually observed from moist tissue surface, which can be a problem in minimally invasive surgeries, where movement of the scope is limited, and the reflections cannot be moved off regions of interest. To provide quantitative evaluation of fluorescence imaging, these undesired reflections need to be removed. Figure 3 shows the pipeline to remove specular reflection. Specular pixels in reflectance channel were first detected using MATLAB function multithresh, which applies Otsu’s algorithm to segment specular reflection. A dilated specular mask was then generated using MATLAB function imdilate with a square structuring element. Since mmSFE captured coregistered reflectance and fluorescence images, the generated specular mask from reflectance would apply to both reflectance and fluorescence channel. Each pixel in dilated specular mask was then replaced by the average of neighbor pixels without specular reflection for both reflectance and fluorescence images. A neighbor size was applied. If the neighbor pixels were all affected by specular reflection, we increased the neighbor size until there were pixels without specular reflection. Fluorescence normalizationThe fluorescence background was previously measured by placing a nonfluorescent white reference 1 cm away from the scope tip with the NIR excitation source, which is later subtracted from raw fluorescence to reject ambient light. To compensate for inconsistent distance between the scope distal tip and various points on the imaged tissue, a ratiometric method is applied. From any point on the tissue surface, the fluorescence and reflectance generated will travel along identical paths to the scope end-face. Therefore, the difference in distance can be compensated using the ratio between these two images. Since mmSFE can capture reflectance and fluorescence images concurrently, there is no registration error between these two channels, which makes this ratiometric method more robust. The method is first tested using a uniform gelatin phantom (10%), mixed with ICG. Gelatin powder (G2500, Sigma-Aldrich) is mixed with distiller warm water (40°C). The gelatin/water mixture is stirred until all the gelatin is fully dissolved, resembling a uniform liquid. ICG (1340009, Sigma-Aldrich) solution is then dispersed in the mixture. The final concentrations of ICG are . An example for this normalization process is shown in Fig. 4. To avoid divide-by-zero errors, each pixel in the reflectance images is added by 1. 2.5.2.Multistep fluorescence registration and two-dimensional tumor map synthesisA multistep image registration procedure was developed and applied to the preprocessed fluorescence image to enable pixel-wise comparisons with stacked histological slides. A surface macrohistological image was first synthesized from 3-D histological slide deck. Then, a multistep registration was then conducted between mmSFE reflectance image and the surface macrohistological image. The transformations calculated from reflectance image were applied to the corresponding fluorescence image. A 2-D tumor map was extracted from histological slides using optical attenuation model and average intensity projection (AIP), which was used as ground truth for comparison. The detailed procedure follows the steps described below. Surface macrohistological image synthesisNIR light penetrates deeper in tissue, which excites the fluorescence beneath tissue surface. To better correlate fluorescence with histological images from both surface and deeper tissues, a 3-D histological slide deck is first generated. Each slide in the 3-D deck is registered to its adjacent slide using MATLAB functions “imregtform” and “imregdemons,” which perform affine transformation and nonrigid registration, respectively. The top-view surface pixels are first extracted from the 3-D histological slide deck and projected to a 2-D plane to synthesize a surface macrohistological image, which is then used to register with mmSFE reflectance image. This surface macrohistological image provides compensation for some incomplete histological slides being cut from uneven tissue surface in paraffin blocks (Fig. 5). Multistep registrationSince mmSFE can capture reflectance and fluorescence images concurrently, there is no spatial registration error between these two images. The surface macrohistological image is registered to the mmSFE reflectance image captured from the 2-mm brain slice, which contains more structural information for registration. In step 1, the boundary of the tissue is identified by leveling the images using MATLAB function “graythresh.” Binary masks of tissue for both reflectance image and macrohistological image are generated. MATLAB function imregtform is used to register these two binary masks for affine transformation. The obtained affine transformation matrix T1 is applied to the fluorescence image captured from the same 2-mm slice using imwarp; in step 2, structural landmarks are selected in both macrohistological image and mmSFE transformed reflectance image from step 1 to serve as control points for a piecewise linear registration using MATLAB function cpselect and a deformed transformation grid T2 is generated using MATLAB function fitgeotrans. The processes of cutting, fixing, and staining of the tissue to create histological slides introduce local deformations, which cannot be corrected using affine transformation. The piecewise linear registration divides the images into triangular meshes, which are individually registered through linear transformations.33 Small spacing of control points allows modeling of local deformation and can address local geometric distortions, whereas large spacing is used to model global deformations. Finally, the deformed grid T2 is applied to the transformed fluorescence image from step 1. The entire pipeline is shown in Fig. 6. Two-dimensional tumor map synthesisA method based on optical attenuation model and AIP was applied to generate a projected 2-D tumor map, which integrated tumor information from different tissue depths. For each digitized H&E-stained image in a 3-D histological slide deck, the tumor profile was outlined by an animal pathologist technician with extensive experience in mouse brain tumor. The outlined tumor profiles were extracted to form a 3-D tumor map deck. Then optical attenuation model was used to simulate attenuated fluorescence emitted from deeper tumor before projecting the tumor maps to the outer surface. The details of these steps are described below. The tumor regions are outlined from the 3-D slide deck, and an optical attenuation model is applied to this 3-D slide deck to compensate the scattering and attenuation in tissue. The effective attenuation coefficient can be calculated as follows: where () is the absorption coefficient equal to 0.1 in biological tissue;34 is the anisotropy factor equal to 0.9,35 and () is the scattering coefficient equal to estimated for cortical layers from 300 to in the NIR range of 830 nm.34The attenuated intensity for each pixel can be calculated as where is the initial intensity of the emitted fluorescence, and d is the distance traveled through the tissue. In this paper, we used () equal to for shallow cortical layers and for depths , measured by Abdo et al.34 in the NIR range of 830 nm. After the attenuation model was applied to 3-D tumor map deck, AIP was used to project attenuated tumor maps to the outer surface, as shown in Fig. 7. The synthesized 2-D tumor map was used as ground truth and compared with the registered fluorescence image captured from the 2-mm slice.2.6.Performance Metric and Statistical AnalysisTo evaluate the proposed pipeline in correlating fluorescence images with histological images and to evaluate BLZ-100 in tumor margin identification, brain slices with tumor and fluorescence labeling were included in the performance analysis. Slices with nonspecific binding fluorescence were excluded. Figure 8 shows an example of nonspecific binding, where BLZ-100 was accumulated in the choroid plexus, which is identifiable from other brain tissues during surgery. For each included slice, the synthesized 2-D tumor map was used as ground truth. An additional region of interest representing the tumor margin was created from the peripheral aspect of the 2-D tumor map using MATLAB function imerode. About 5000 pixels, accounting for in area, were randomly selected from the tumor core, tumor margin, and background regions, enabling objective comparisons of the fluorescence. Target-to-background (T/B) ratio was then calculated in the tumor core and margin for each slice. Fluorescence intensity was used for fluorescence image thresholding to delineate tumor boundary. A pixel-wise comparison between 2-D tumor map and thresholded fluorescence image was conducted. The optimal threshold was determined using Youden’s index of the receiver operating characteristic (ROC) curve,36 which optimizes the sum of sensitivity and specificity. Sensitivity, specificity, and DSC, which is a popular metric for evaluating image segmentation and registration,37 were calculated accordingly. The AUC of ROC curve was performed to determine the accuracy to differentiate between normal tissue and tumor, which evaluates the proposed methods independently of the threshold. To evaluate the performance of using synthesized 2-D tumor map as ground truth, the average sensitivity, specificity, DSC, and AUC were calculated and compared to those of using selected best slide (first complete slide in the 3-D slide deck) and randomly selected slide from the 3-D slide deck as ground truth. Group averages and variances were analyzed using paired samples -tests and -test to assess differences in the performance using stacked slides, selected best slide, and randomly selected slide as ground truth, in terms of the accuracy in differentiating between tumor and normal brain. 3.Results3.1.Distance Compensation PerformanceThe proposed method, to compensate for inconsistent distance between the scope and the tissue, was tested using a uniform gelatin phantom. The mmSFE scope tip was placed perpendicular to the phantom at 1 cm away. Fluorescence and reflectance images were captured and combined as a ratio image, as shown in Figs. 9(a)–9(c). A 20 pixel-wide line profile was generated for each image at the same coordinates. The average intensity of the 20 pixels was plotted along the profile, which showed large variations in fluorescence (blue) and reflectance (red). The fluorescence and reflectance intensity at the image center was much greater than that at the periphery. The combination of reflectance and fluorescence as a ratio image (yellow) reduced these variations along the profile [Figs. 9(c)–9(d)]. 3.2.Multistep Image RegistrationThe surface macrohistological image was created from 3-D H&E slide deck and registered to calibrated mmSFE reflectance image from 2-mm slice. Affine transformation was applied to correct global deformations, such as scale difference, orientation difference, while piecewise linear registration was applied to correct local deformations. Figure 10 shows examples of registration results: original mmSFE reflectance image before registration (a, f, k); mmSFE green fluorescence image overlaid on reflectance image after registration (b, g, l); thresholded mmSFE fluorescence image overlaid on reflectance image after registration (c, h, m); synthesized 2-D green tumor map overlaid on grayscale macrohistological image (d, i, n); and manually delineated tumor map overlaid on selected best slide (e, j, o). Alpha blending was applied to overlay fluorescence image over reflectance, which maintains the structural information present in the reflectance while adding the molecular information of the fluorescence distribution. As can be seen in Fig. 10(n), the synthesized 2-D tumor map projected deeper tumor directly onto the surface. Tumor map from selected slide may underestimate the tumor information, as seen in Figs. 10(e), 10(j), 10(o). Fluorescence hotspot colocalized well with the tumor regions on the synthesized 2-D tumor map. However, the spatial distribution of BLZ-100 was not uniform. BLZ-100 showed strong accumulation in tumor core, while the margin appeared more diffuse, as seen in Figs. 10(b), 10(g), 10(l). 3.3.Overall Framework Performance and Statistical AnalysisA total of 50 brain slices were used for mmSFE fluorescence imaging. Four slices became incomplete during the histological process. The confusion matrix of the remaining 46 slices is shown in Table 1. The number of slices with tumor that were successfully labeled using BLZ-100 was 29. These true-positive cases were used to evaluate this fluorescence imaging agent in tumor margin identification. For true negatives, six slides had no tumor present and displayed no fluorescence label. Eight slides with nonspecific BLZ-100 binding in the choroid plexus were excluded in performance analysis. Three false-negative cases were found, where little or no fluorescence was observed in slices with tumor. Table 1Confusion matrix of BLZ-100 performance in labeling medulloblastoma tumor model (Med-114FH).
For the 29 included true-positive cases, T/B ratios for the tumor core and margin were calculated using 5000 different points () randomly selected from tumor core, margin, and normal regions. The mean T/B ratios () for tumor core and margin were and , respectively, with -value determined by paired -test. Fluorescence intensity was used to threshold tumor boundary, and ROC curve was generated accordingly. ROC curve analysis showed marked difference in the performance among using integrated stacked slides as ground truth, using selected best single slide as ground truth, and randomly selected single slide as ground truth, in terms of differentiating between tumor and healthy tissue. Mean sensitivity, specificity, DSC, and AUC of ROC curve were investigated to compare the methods to generate tumor map. The synthesized 2-D tumor map performed better than the selected best slide and randomly selected slide, as shown in Table 2 and Fig. 11. The average DSC for synthesized 2-D tumor map was significantly greater than that for selected best slide (0.582 versus 0.398 with -value ) and that for randomly selected slide (0.582 versus 0.396 with -value ). The average AUC for synthesized tumor map was greater than that for selected best slide (88% versus 85.5%), but there was no statistically significant improvement. The average AUC for synthesized tumor map was significantly greater than that for randomly selected slide (88% versus 84.1% with -value ). The variance of the AUC using synthesized tumor map as ground truth was significantly less than that using selected slide as ground truth (0.057 versus 0.087, -value 0.0145) and that using randomly selected slide as ground truth (0.057 versus 0.085, -value 0.0212). Table 2Mean sensitivity, specificity, DSC and AUC in differentiating between normal tissue and tumor, compared among the synthesized 2-D tumor map, selected best slide, and randomly selected slide.
Note: The 95% CI of the mean is indicated in the parentheses. 4.Discussion and ConclusionThe overall goal of this paper is to develop a pipeline to evaluate the performance of fluorescence imaging agents in terms of tumor identification. This work is divided into two parts, first the fluorescence image is calibrated for accurate fluorescence quantification. As stated in Ref. 28, the performance of fluorescence imaging is affected by the variable experimental settings, including lesion depth and camera–tissue distance. To connect fluorescence imaging to clinical outcome and to ensure a reproducible performance, calibration of fluorescence imaging is required. To address this problem, a specular reflectance rejection algorithm is applied, and a background subtraction method is used to reject ambient light. Inconsistent distance and angle is then compensated by combining reflectance image and fluorescence as a ratio image. When testing in the gelatin phantom, the ratiometric method compensated nonuniform fluorescence. As can be seen on the intensity plot (Fig. 9), fluorescence is much higher in the center of the image than at the periphery. The ratio image corrected for these differences over the width of the image and the uniform fluorescence (80% to 100% of the maximum fluorescence) is extended from 50% to 80%. Second, a multistep registration framework is established to correlate fluorescence images to stacked histological images. To preclinically evaluate an imaging agent, a proper registration between the fluorescence and histological images is needed. However, current registration methods compare to single selected H&E slide,26,27 which is subjective and may underestimate fluorescence from deeper lesions, especially for NIR fluorescence agents. In this paper, an optical attenuation model and AIP are applied to integrate multiple histological slides and project deeper tumor regions to the surface. One advantage of this proposed method is the use of stacked H&E slides from surface to deeper tissues, to generate a synthesized 2-D tumor map, which provides an approximation of light in “seeing” tumor from deeper tissue. Then global and local deformations are corrected using affine transformation and piecewise linear registration. Structural landmarks from the reflectance image are selected as control points for piecewise linear registration. NIR agent, BLZ-100, is used to evaluate the proposed pipeline. The 2-D synthesized tumor map is compared to the selected slide as ground truth. In this study, we select the complete slide with the best quality and a random slide in the 3-D slide deck as comparison. Sensitivity, specificity, DSC, and AUC are calculated as evaluation metrics. The optimal fluorescence intensity threshold for tumor delineation is calculated based on Youden’s index, which optimizes the sum of sensitivity and specificity. There is no significant difference in sensitivity and specificity when comparing the three methods. However, we obtain a significantly higher DSC when using the synthesized tumor map as ground truth. The reasons are illustrated as follows: (1) the selected slide only accounts for a single plane from the 3-D tumor, which underestimates the tumor information. The underestimated tumor map results in a reduced number of true-positive pixels and an increased number of negative pixels, as seen in Figs. 10(e), 10(j), 10(o). In contrast, there are more true-positive pixels and fewer negative pixels when using synthesized 2-D tumor map as ground truth. (2) The imbalanced pixel number between normal tissue and tumor aggravates the discrepancies in the ratio between the number of negative pixels and the number of true-positive pixels (synthesized tumor map versus selected best slide: 7.42 versus 18.57, with -value determined by paired -test). The significant discrepancies in the ratio between the number of negative pixels and the number of true-positive pixels result in a significant difference in DSC, as shown in the equation where is the number of negative pixels and TP is the number of true-positive pixels. This imbalance between the number of true-positive pixels and the number of negative pixels diminishes the robustness of using Youden’s index to select optimal threshold. AUC is a more reliable metric to evaluate the proposed methods, which is independent of the selected threshold. The higher average DSC and AUC suggests a better spatial overlap between BLZ-100 fluorescence and synthesized tumor map from H&E slides. Especially, the proposed method outperforms the randomly selected single plane, which is usually obtained from regular pathological processes, with significantly higher DSC and AUC. When compared to using selected best slide and randomly selected slide as ground truth, the significantly smaller variance of the AUC using synthesized 2-D tumor map as ground truth indicates that this method is less biased, more stable, and more reproducible. Overall, the 2-D synthesized tumor map outperforms the selected best slide and randomly selected slide when comparing with the NIR fluorescence image, which is also a 2-D mapping of a 3-D tumor.The experiment is conducted on ex-vivo tissue as preclinical evaluation. The performance of this pipeline needs to be investigated for in-vivo intraoperative imaging, which can be challenged by the complex tissue properties and the variable experimental arrangement. For example, we fixed the mmSFE position with one perspective for tissue imaging in this study. During freehand surgical applications, mmSFE position will be adjusted for multiperspective views in real-time video. The change of the scope–tissue angle may diminish the importance of the proposed distance compensation method. One assumption of the proposed method is that the fluorescence and reflectance light travels along identical paths to the scope, which is valid for neoplastic lesions topically labeled from the epithelial layer.22 However, systemic labeling of lesions using NIR fluorescence can emit from deeper tumors, which makes the light paths not identical to the scope. A tilted orientation of scope-to-tissue surface may magnify this error. On the other hand, the multiperspective views can be used for elimination of specular reflections without loss of information. Current specular removal method uses surrounding pixels to replace high-intensity pixels from specular reflection, which only corrects high intensity but neglects the original structural features beneath the specular reflection. Future work is needed to design new specular reflection rejection algorithm, which corrects specular reflection using the same region from a different perspective view. One limitation of this study is that only the 29 slices with tumor and fluorescence labeling are included in the performance analysis, which limits a comprehensive evaluation of BLZ-100 in tumor identification. However, we evaluate the proposed pipeline in correlating fluorescence images to stacked histological images instead of testing the clinical effectiveness of BLZ-100. In addition, five H&E slices are integrated to generate a synthesized 2-D tumor map, which accounts for only 0.5 mm in depth. Depending on the type of tissue, the penetration depth of light is at 400 nm and increases to about 5 mm at 700 to 900 nm using skin as an example.38 The NIR fluorescence detection depth of the mmSFE is measured using ICG solution from to 100 nM diluted in dimethyl sulfoxide. ICG solutions from to 100 nM in microhematocrit capillary tubes (Fisher Scientific, 1.1 mm ID) are buried in NSG mouse brain (0.5, 1, 2, and 3 mm from brain surface). The mmSFE is used to capture the ICG fluorescence at 1 cm away from the brain surface. T/B ratio is calculated accordingly. The mmSFE detection depth is shown in Table 3. From the table, we can see that mmSFE can detect 100 nM ICG at a depth of 0.5 mm and ICG at a depth of 1 mm in mouse brain tissue. Since the estimated BLZ-100 dye concentration accumulated in the tumor is up to the equivalent of , applying five layers of H&E slices (total depth of 0.5 mm) to synthesize the 2-D tumor map is underestimated. To better represent the information from deeper lesions detected by the NIR probes, more histological slices, 1 mm in total depth, should be integrated as a synthesized 2-D tumor map and compared with the mmSFE NIR fluorescence image. Moreover, the optical attenuation model is limited by the assumption of straight-line fluorescence attenuation (orthogonal to tissue surface) which is incorrect for highly scattering brain tissue. More complex models, such as Monte Carlo light scattering simulation, can be used to better simulate fluorescence scattering in tissue.39 In contrast, maximal intensity projection (MIP) can be regarded as a simplified model, which has similar results compared to the optical attenuation model. However, the 2-D tumor map synthesized using MIP does not include tumor depth information. Instead, the optical attenuation model and Monte Carlo light scattering simulate the fluorescence scattering in the tissue, which can provide more realistic tumor maps, including tumor depth information. Last but not least, we used single NIR imaging agent for one type of tumor detection in one mouse model. The efficiency of the proposed pipeline in calibrating different imaging agents to histology for different tumor types needs to be further investigated. In our future work, we aim to automate the whole pipeline, thus providing reliable tool to evaluate fluorescent probes in delineating margins. The calibration techniques can be realized in real time and implemented using mmSFE for standardized in vivo fluorescence imaging, along with preprocedure and possibly postprocedure calibrations. The multistep registrations can be automated using laser-induced fiducial landmarks40 and algorithm-selected landmarks.41 Table 3The mmSFE detection depth measured in T/B ratio.
Note: — indicates that fluorescence signal cannot be differentiated from noise. When BLZ-100 NIR fluorescence is compared to three other fluorescence agents in Elliott et al.,26 in general, their performance in detecting glioma tumor area is comparable. In terms of mean accuracy determined by ROC curve analysis for wild-type tumor, BLZ-100 performed slightly better than the single fluorescence agents (BLZ-100: 88% versus IRDye680: 77% versus PpIX: 87% versus ABY029: 69%), but slightly worse than the combination of these agents (combination: 90%).26 In terms of enhanced fluorescence contrast, BLZ-100 accumulates in both tumor cores and tumor margins with mean T/B ratios of 8.64 and 4.82, respectively. The tumor core showed 1.8-fold higher T/B ratio values compared with tumor margin (-value determined by paired -test). The margin fluorescence appears more diffuse, which could be scattering fluorescence from deeper tumor lesions into nontumor regions closer to the surface. While PpIX shows weak contrast in tumor margins and appears to contain more high spatial frequency features, sharper edges, and patterns, IRDye680RD enhances the necrotic regions provided by the EPR effect and ABY-029 shows enhanced fluorescence in both tumor core and margin but limits to epidermal growth factor receptor (EGFR) overexpressed tumor type.26 As can be seen, these imaging agents can provide complementary properties (metabolism, perfusion, and EGFR expression) and may be combined with BLZ-100 in the future to provide more accurate tumor contrast for guiding surgery.42 For example, in addition to the systemically applied BLZ-100, the QRH-882260 + Cy5 peptide43 can be topically applied intraoperatively to the residual tumor during surgery to provide better visualization of tumor margins. The adoption of multiagent strategy for more accurate tumor delineation can be achieved by using mmSFE, which can detect multiple spectral bands of fluorescence concurrently. There is essentially no registration error across fluorescence channels, which makes multiagent imaging using mmSFE more powerful. Moreover, mmSFE can be modified for an additional OCT channel.44 The combination of en face fluorescence imaging and 3-D morphological imaging allows molecular and structural contrasts simultaneously, which has been recently demonstrated using a xenograft mouse model of human colorectal cancer ex vivo.45 DisclosuresEJS is a coinventor of the scanning fiber endoscope and has participated in a royalty sharing program at the University of Washington. AcknowledgmentsFunding has been provided by NIH R01 EB016457 (PI: Seibel) in collaboration with the Dr. James Olson Lab, Fred Hutchinson Cancer Research Center, Seattle, Washington, by NIH R01 CA200007, U54 CA163059 (PI: Wang & Seibel) for fluorescence early cancer detection, and by NCI SBIR R43 CA211086 (PI: McMorrow), which supports the joint effort between the Human Photonics Lab at the University of Washington and VerAvanti Inc., Redmond, Washington. BLZ-100 has been kindly provided by Blaze Bioscience Inc., Seattle, Washington. ReferencesM. Lacroix et al.,
“A multivariate analysis of 416 patients with glioblastoma multiforme: prognosis, extent of resection, and survival,”
J. Neurosurg., 95
(2), 190
–198
(2001). https://doi.org/10.3171/jns.2001.95.2.0190 JONSAC 0022-3085 Google Scholar
M. J. McGirt et al.,
“Independent association of extent of resection with survival in patients with malignant brain astrocytoma,”
J. Neurosurg., 110
(1), 156
–162
(2009). https://doi.org/10.3171/2008.4.17536 JONSAC 0022-3085 Google Scholar
W. Stummer et al.,
“Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,”
Lancet Oncol., 7
(5), 392
–401
(2006). https://doi.org/10.1016/S1470-2045(06)70665-9 Google Scholar
K. S. Samkoe et al.,
“Application of fluorescence-guided surgery to subsurface cancers requiring wide local excision: literature review and novel developments toward indirect visualization,”
Cancer Control, 25
(1), 107327481775233
(2018). https://doi.org/10.1177/1073274817752332 Google Scholar
T. E. Grotz et al.,
“Mayo Clinic consensus recommendations for the depth of excision in primary cutaneous melanoma,”
Mayo Clin. Proc., 86
(6), 522
–528
(2011). https://doi.org/10.4065/mcp.2011.0059 MACPAJ 0025-6196 Google Scholar
N. L. Martirosyan et al.,
“Use of in vivo near-infrared laser confocal endomicroscopy with indocyanine green to detect the boundary of infiltrative tumor,”
J. Neurosurg., 115
(6), 1131
–1138
(2011). https://doi.org/10.3171/2011.8.JNS11559 JONSAC 0022-3085 Google Scholar
Z. Hamdoon et al.,
“Optical coherence tomography in the assessment of oral squamous cell carcinoma resection margins,”
Photodiagn. Photodyn. Ther., 13 211
–217
(2016). https://doi.org/10.1016/j.pdpdt.2015.07.170 Google Scholar
H. Fabelo et al.,
“An intraoperative visualization system using hyperspectral imaging to aid in brain tumor delineation,”
Sensors, 18
(2), 430
(2018). https://doi.org/10.3390/s18020430 SNSRES 0746-9462 Google Scholar
L. Jiang et al.,
“Label-free imaging of brain and brain tumor specimens with combined two-photon excited fluorescence and second harmonic generation microscopy,”
Laser Phys. Lett., 14
(10), 105401
(2017). https://doi.org/10.1088/1612-202X/aa7c9a 1612-2011 Google Scholar
H. Karabeber et al.,
“Guiding brain tumor resection using surface-enhanced Raman scattering nanoparticles and a hand-held Raman scanner,”
ACS Nano, 8
(10), 9755
–9766
(2014). https://doi.org/10.1021/nn503948b ANCAC3 1936-0851 Google Scholar
Y. Jiang et al.,
“Ultrathin and flexible 4-channel scope for guiding surgical resections using a near-infrared fluorescence molecular probe for cancer,”
Proc. SPIE, 10576 105762K
(2018). https://doi.org/10.1117/12.2292092 PSISDG 0277-786X Google Scholar
E. Hartmans et al.,
“Potential red-flag identification of colorectal adenomas with wide-field fluorescence molecular endoscopy,”
Theranostics, 8
(6), 1458
–1467
(2018). https://doi.org/10.7150/thno.22033 Google Scholar
S. J. Erickson-Bhatt et al.,
“Real-time imaging of the resection bed using a handheld probe to reduce incidence of microscopic positive margins in cancer surgery,”
Cancer Res., 75
(18), 3706
–3712
(2015). https://doi.org/10.1158/0008-5472.CAN-15-0464 Google Scholar
F. Acerbi et al.,
“Fluorescein-guided surgery for grade IV gliomas with a dedicated filter on the surgical microscope: preliminary results in 12 cases,”
Acta Neurochir., 155
(7), 1277
–1286
(2013). https://doi.org/10.1007/s00701-013-1734-9 Google Scholar
X. Wang et al.,
“Compact instrument for fluorescence image-guided surgery,”
J. Biomed. Opt., 15
(2), 020509
(2010). https://doi.org/10.1117/1.3378128 JBOPFO 1083-3668 Google Scholar
J. Y. Lee et al.,
“Near-infrared fluorescent image-guided surgery for intracranial meningioma,”
J. Neurosurg., 128
(2), 380
–390
(2018). https://doi.org/10.3171/2016.10.JNS161636 JONSAC 0022-3085 Google Scholar
B. K. Hendricks, N. Sanai and W. Stummer,
“Fluorescence-guided surgery with aminolevulinic acid for low-grade gliomas,”
J. Neuro-oncol., 141
(1), 13
–18
(2018). https://doi.org/10.1007/s11060-018-03026-6 JNODD2 0167-594X Google Scholar
W. Stummer et al.,
“Fluorescence-guided resection of glioblastoma multiforme utilizing 5-ALA-induced porphyrins: a prospective study in 52 consecutive patients,”
J. Neurosurg., 93
(6), 1003
–1013
(2000). https://doi.org/10.3171/jns.2000.93.6.1003 JONSAC 0022-3085 Google Scholar
E. Belykh et al.,
“Scanning fiber endoscope improves detection of 5-aminolevulinic acid-induced protoporphyrin IX fluorescence at the boundary of infiltrative glioma,”
World Neurosurg., 113 e51
–e69
(2018). https://doi.org/10.1016/j.wneu.2018.01.151 Google Scholar
E. L. Rosenthal et al.,
“Safety and tumor specificity of cetuximab-IRDye800 for surgical navigation in head and neck cancer,”
Clin. Cancer Res., 21
(16), 3658
–3666
(2015). https://doi.org/10.1158/1078-0432.CCR-14-3284 Google Scholar
A. L. R. de Souza et al.,
“Fluorescent affibody molecule administered in vivo at a microdose level labels EGFR expressing glioma tumor regions,”
Mol. Imaging Biol., 19
(1), 41
–48
(2017). https://doi.org/10.1007/s11307-016-0980-7 Google Scholar
B. P. Joshi et al.,
“Multimodal endoscope can quantify wide-field fluorescence detection of Barrett’s neoplasia,”
Endoscopy, 48
(2), A1
–A13
(2016). https://doi.org/10.1055/s-0034-1392803 ENDCAM Google Scholar
R. R. Zhang et al.,
“Beyond the margins: real-time detection of cancer using targeted fluorophores,”
Nat. Rev. Clin. Oncol., 14
(6), 347
–364
(2017). https://doi.org/10.1038/nrclinonc.2016.212 Google Scholar
P. V. Butte et al.,
“Near-infrared imaging of brain tumors using the tumor paint BLZ-100 to achieve near-complete resection of brain tumors,”
Neurosurg. Focus, 36
(2), E1
(2014). https://doi.org/10.3171/2013.11.FOCUS13497 Google Scholar
J. Fidel et al.,
“Preclinical validation of the utility of BLZ-100 in providing fluorescence contrast for imaging spontaneous solid tumors,”
Cancer Res., 75
(20), 4283
–4291
(2015). https://doi.org/10.1158/0008-5472.CAN-15-0471 Google Scholar
J. T. Elliott et al.,
“Simultaneous in vivo fluorescent markers for perfusion, protoporphyrin metabolism, and EGFR expression for optically guided identification of orthotopic glioma,”
Clin. Cancer Res., 23
(9), 2203
–2212
(2017). https://doi.org/10.1158/1078-0432.CCR-16-1400 Google Scholar
G. Lu et al.,
“Hyperspectral imaging for cancer surgical margin delineation: registration of hyperspectral and histological images,”
Proc. SPIE, 9036 90360S
(2014). https://doi.org/10.1117/12.2043805 PSISDG 0277-786X Google Scholar
M. Koch, P. Symvoulidis and V. Ntziachristos,
“Tackling standardization in fluorescence molecular imaging,”
Nat. Photonics, 12 505
–515
(2018). https://doi.org/10.1038/s41566-018-0221-5 NPAHBY 1749-4885 Google Scholar
S. Brabetz et al.,
“A biobank of patient-derived pediatric brain tumor models,”
Nat. Med., 24
(11), 1752
–1761
(2018). https://doi.org/10.1038/s41591-018-0207-3 1078-8956 Google Scholar
J. Parrish-Novak et al.,
“Nonclinical profile of BLZ-100, a tumor-targeting fluorescent imaging agent,”
Int. J. Toxicol., 36
(2), 104
–112
(2017). https://doi.org/10.1177/1091581817697685 Google Scholar
C. M. Lee et al.,
“Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,”
J. Biophotonics, 3
(5-6), 385
–407
(2010). https://doi.org/10.1002/jbio.v3:5/6 Google Scholar
S. J. Miller et al.,
“Special section on endomicroscopy technologies and biomedical applications: Targeted detection of murine colonic dysplasia in vivo with flexible multispectral scanning fiber endoscopy,”
J. Biomed. Opt., 17
(2), 021103
(2012). https://doi.org/10.1117/1.JBO.17.2.021103 JBOPFO 1083-3668 Google Scholar
A. Goshtasby,
“Piecewise linear mapping functions for image registration,”
Pattern Recognit., 19
(6), 459
–466
(1986). https://doi.org/10.1016/0031-3203(86)90044-0 Google Scholar
A. Abdo, A. Ersen and M. Sahin,
“Near-infrared light penetration profile in the rodent brain,”
J. Biomed. Opt., 18
(7), 075001
(2013). https://doi.org/10.1117/1.JBO.18.7.075001 JBOPFO 1083-3668 Google Scholar
M. Johns et al.,
“Determination of reduced scattering coefficient of biological tissue from a needle-like probe,”
Opt. Express, 13
(13), 4828
–4842
(2005). https://doi.org/10.1364/OPEX.13.004828 OPEXFF 1094-4087 Google Scholar
Y. Jiang et al.,
“Toward real-time quantification of fluorescence molecular probes using target/background ratio for guiding biopsy and endoscopic therapy of esophageal neoplasia,”
J. Med. Imaging, 4
(2), 024502
(2017). https://doi.org/10.1117/1.JMI.4.2.024502 JMEIET 0920-5497 Google Scholar
L. R. Dice,
“Measures of the amount of ecologic association between species,”
Ecology, 26
(3), 297
–302
(1945). https://doi.org/10.2307/1932409 ECGYAQ 0094-6621 Google Scholar
D. Barolet,
“Light-emitting diodes (LEDs) in dermatology,”
Semin. Cutan Med. Surg., 27
(4), 227
–238
(2008). https://doi.org/10.1016/j.sder.2008.08.003 Google Scholar
A. J. Welch et al.,
“Propagation of fluorescent light,”
Lasers Surg. Med., 21
(2), 166
–178
(1997). https://doi.org/10.1002/(SICI)1096-9101(1997)21:2<166::AID-LSM8>3.0.CO;2-O LSMEDI 0196-8092 Google Scholar
J. Unger et al.,
“Method for accurate registration of tissue autofluorescence imaging data with corresponding histology: a means for enhanced tumor margin assessment,”
J. Biomed. Opt., 23
(1), 015001
(2018). https://doi.org/10.1117/1.JBO.23.1.015001 JBOPFO 1083-3668 Google Scholar
C. W. Wang, S. M. Ka and A. Chen,
“Robust image registration of biological microscopic images,”
Sci. Rep., 4 6050
(2014). https://doi.org/10.1038/srep06050 SRCEC3 2045-2322 Google Scholar
B. W. Pogue et al.,
“Perspective review of what is needed for molecular-specific fluorescence-guided surgery,”
J. Biomed. Opt., 23
(10), 100601
(2018). https://doi.org/10.1117/1.JBO.23.10.100601 JBOPFO 1083-3668 Google Scholar
J. Zhou et al.,
“EGFR overexpressed in colonic neoplasia can be detected on wide-field endoscopic imaging,”
Clin. Transl. Gastroenterol., 6
(7), e101
(2015). https://doi.org/10.1038/ctg.2015.28 Google Scholar
K. L. Lurie et al.,
“Rapid scanning catheterscope for expanded forward-view volumetric imaging with optical coherence tomography,”
Opt. Lett., 40
(13), 3165
–3168
(2015). https://doi.org/10.1364/OL.40.003165 OPLEDP 0146-9592 Google Scholar
F. Feroldi et al.,
“High resolution combined molecular and structural optical imaging of colorectal cancer in a xenograft mouse model,”
Biomed. Opt. Express, 9
(12), 6186
–6204
(2018). https://doi.org/10.1364/BOE.9.006186 BOEICL 2156-7085 Google Scholar
BiographyYang Jiang received his MS degree in biomedical engineering at Duke University. He is working on a PhD in bioengineering at University of Washington. His research interest is using computer vision and machine learning techniques for medical imaging analysis. Emily J. Girard is a staff scientist at the Fred Hutchinson Cancer Research Center in the lab of James Olson MD PhD. She is an in vivo pharmacologist with experience in the biotech industry and in academic research. Her research focus is on identification of novel and repurposing of approved therapeutics for pediatric brain tumors. Fiona Pakiam graduated from Edinburgh Napier University with a Bachelor of Science and gained honors from the University of Strathclyde. With over ten years of experience as a board certified histologist by the American Society for Clinical Pathology, she has worked in both research and clinical settings. Eric J. Seibel is a research professor of mechanical engineering, adjunct in departments of bioengineering, electrical and computer engineering, oral health sciences, and director of the Human Photonics Laboratory at the University of Washington. He received his BS and MS degrees in mechanical engineering from Cornell University and University of California, Berkeley, respectively, and his PhD in bioengineering from the University of Washington, Seattle. He invented the scanning fiber endoscopy and developed instrumentation techniques for early disease diagnosis and image-guided interventions. He is a senior member of SPIE. |