2.1.

Grayscale Imaging Mode

Figure 1 shows the layout of the confocal microendoscope in grayscale mode.¹¹ A $488\text{-}\mathrm{nm}$ diode-pumped solid-state laser (Coherent, Santa Clara, California) is used as the excitation source. A line of illumination, formed by spherical and cylindrical lenses, is coupled into the proximal end of the catheter by a $10\times$ microscope objective (Leeds Precision Instruments, Minncapolis, Minnesotre). A 30,000 element coherent fiber bundle (Sumitomo Electric USA, Torrance, California) transmits the illumination profile to the distal end of the fiber, which is imaged into the tissue by a miniature objective. A focus mechanism enables focus control down to $200\mu \mathrm{m}$ below the tissue surface. Scan mirror 1 sweeps the illumination profile across the field of view. Fluorescence light from the sample is collected back into the system via the miniature objective and fiber bundle, and separated from the reflected illumination by a dichroic beamsplitter (Chroma Technology Corporation, Rockingham, Vermont). The fluorescence light is then focused onto the confocal slit aperture, which rejects light from out-of-focus planes and provides the optical sectioning (confocal) capability of the system. A second scan mirror, placed between two lenses, images the light passing through the confocal slit onto a charge-coupled device (CCD) camera (Photometrics, Tucson, Arizona). The two scan mirrors are synchronized to provide correspondence between the illumination profile in the tissue and the detection profile on the CCD. Figure 2 shows a picture of the confocal microendoscope system.¹¹

Fig. 1

Layout of the confocal microendoscope. The orthogonal view shows the illumination path from the direction for which the cylindrical lens has optical power.

Fig. 2

Photograph of the confocal microendoscope.

In operation, the distal end of the catheter is placed in contact with the sample and a small amount of fluorescent dye ( $\approx 10\mu \mathrm{L}$ per image) is delivered to the area of interest. The user adjusts the focus of the catheter, and images are collected by the computer for processing and/or subsequent analysis.

2.2.

Multispectral Imaging Mode

2.2.1.

Layout of the multispectral channel

The layout of the multispectral mode is shown in Fig. 3 . The novel design requires nothing more than a $45\text{-}\mathrm{deg}$ BK7 prism (OptoSigma, Santa Ana, California) located away from the optical axis. The off-axis position enables us to leave the prism in a fixed position that does not disturb the beam path in grayscale mode. The system switches rapidly between the two modes of operation (grayscale and multispectral) in less than half a second. To switch from grayscale to multispectral imaging mode, the computer-controlled galvo mirror (scan mirror 2) is rotated to a fixed position. The fluorescence signal, reflected by this mirror, is dispersed by the prism and focused on the CCD camera at different spatial locations according to wavelength. For each position of the first scan mirror (scan mirror 1), the system records the 2-D light signal pattern on the CCD, which has one spatial dimension (along the slit direction) and one spectral dimension (perpendicular to the slit direction). Repeating this process (reading out a frame of data from the 2-D detector) for every position of the first scan mirror allows one to reconstruct a 3-D data cube that consists of two spatial dimensions and one spectral dimension. The prism is adjusted for minimum deviation to simplify the design and the theoretical modeling of the system. This configuration also yields a simple way of aligning the prism in the system. The orientation of the prism and the incidence angle of the beam on the prism entrance face are such that the central wavelength, $600\mathrm{nm}$ , is imaged to the center of the CCD detector at minimum deviation. In other words, the $600\text{-}\mathrm{nm}$ signal comes out of the prism parallel to the optical axis. The diameter of the camera lens ( $180\text{-}\mathrm{mm}$ focal length and $60\text{-}\mathrm{mm}$ entrance pupil) is large enough to collect the dispersed beam from the prism.

Fig. 3

Grayscale (a) and multispectral (b) beam paths. Shorter wavelengths exit the prism with the largest deviation angle. Dispersed light is focused on the CCD camera. The position on the detector is directly related to wavelength. Mirror 2 is stationary in the multispectral mode of operation.

A calibration procedure employing a theoretical dispersion profile is used to map detector pixels to wavelength. The details of the calibration procedure are discussed in the next section.

2.2.2.

Tools used in the multispectral mode

Multspectral data cube

As discussed in the previous section, the camera records one spatial dimension and one spectral dimension for each orientation of scan mirror 1. Figure 4 shows images obtained from three different positions of scan mirror 1. The amplitude of the mirror is set to sweep the illumination line over the full field of the fiberoptic imaging bundle. The frequency of the mirror is adjusted so that the camera acquires 512 frames in a single sweep. Collection of a full dataset takes roughly $6\mathrm{s}$ and results in a data cube consisting of $512\times 512$ spatial pixels and 150 spectral pixels.

Fig. 4

Relationship between spatial location (top graphs) and spectral frames (bottom graphs). The lines in the upper images represent the slit illumination profile as it scans across the $y$ axis of the field of view. The lower images show the dispersion introduced by the prism at the selected scanning positions.

Display software

Custom software was developed to aid in the display and analysis of the 3-D multispectral data cube. Figure 5 shows a screenshot of this software when the confocal microendoscope was back illuminated by a white light source filtered down to a $695\text{-}\text{to}705\text{-}\mathrm{nm}$ spectral range. The main window shows the 2-D spatial distribution of the sample at a specific wavelength, while the side windows show the distribution of one spatial dimension as a function of wavelength. At the bottom, the software displays the full spectrum associated with the current cursor location.

Fig. 5

Image of the illuminated fiber bundle using a lamp with a $700\text{-}\mathrm{nm}$ optical filter. (a) Cross-section planes of the data cube and (b) spectrum of the image point indicated by a white cross. The intensity is unitless because it is scaled relatively to the maximum intensity in the image.

The program is fully interactive; a mouse click anywhere on the 2-D image in the main window will plot the spectrum associated with that point and update the side windows to show the spectral information along the horizontal and vertical lines through the selected point. A mouse click on one of the side windows will display, in the main window, the 2-D spatial distribution of the 3-D data cube at the specified wavelength.

Wavelength calibration procedure

To plot the spectrum associated with a given spatial point in the image (bottom plot of the display program), it is necessary to determine the relationship between CCD pixel position and wavelength.

The fixed tilt angle ${\theta }_m$ of mirror 2 and the rotation angle ${\theta }_p$ of the prism define the incident angle ${\theta }_i$ of the light on the prism surface. Given the incident angle ${\theta }_i$ of light on a prism of wedge angle $\alpha$ and index of refraction $n$ , the deviation angle $D$ produced by the prism is given by

Eq. 1

$$D\left(\lambda \right)={\theta }_i-\alpha +{\sin }^{-1}(\sin \alpha \sqrt{n{\left(\lambda \right)}^2-{\sin }^2{\theta }_i}-\sin {\theta }_i\cos \alpha ).$$

The deviation angle varies with wavelength because the index of refraction of the prism is wavelength dependent. This property is well described by the Sellmeier equation for the BK7 glass in the prism. The deviation $D$ and the angles ${\theta }_m$ and ${\theta }_p$ are related to the incident angle ${\theta }_c$ of the light entering the camera. The pixel offset on the CCD is given by the relationship

Eq. 2

$$Np=f_c\tan {\theta }_c,$$

where $N$ is the number of pixels offset from the optical axis, $p$ is the pixel size, $f_c$ is the focal length of the camera lens, and ${\theta }_c$ is the angle of incidence on the camera lens. Using Eq. 2, the Sellmeier equation, and the various parameters of the system, it is possible to numerically invert Eq. 1 to determine wavelength as a function of pixel offset or $\lambda \left(N\right)$ . Through this numerical inversion, a lookup table can be generated to precisely convert CCD pixel offset to wavelength.

In practice, a multispectral calibration image is collected using a narrowband optical filter (e.g., Fig. 5). The calibration routine looks for the maximum intensity in the reference 3-D data cube and records its spectral value in terms of pixel number on the CCD. The program then uses this information, along with a pixel offset lookup table, derived by the method just described, to determine the correspondence between wavelength and pixel position for the entire CCD.

3.1.

Performance of the System

The slit-scan confocal microendoscope uses a catheter based on a fiber optic imaging bundle. The pixilated fiber bundle acts as a sampling system with a $3\text{-}\mu \mathrm{m}$ center-to-center pixel spacing. In terms of the spatial resolution, the effect of the fiber bundle may be modeled as a convolution of the object with the pixel function. This model predicts a theoretical tissue-space lateral resolution of $1.9\mu \mathrm{m}$ and a theoretical tissue-space axial resolution of $20\mu \mathrm{m}$ using our $3\text{-}\mathrm{mm}$ -diam distal objective with a magnification of 1.5 from tissue to fiber.¹³ The confocal microendoscope provides the same spatial resolution in both multispectral and grayscale imaging modes. Spatial resolution was measured, based on the full width at half maximum (FWHM) of the point spread function (PSF), by Rouse ¹⁰ to be roughly $3\text{-}\mu \mathrm{m}$ lateral and $30\text{-}\mu \mathrm{m}$ axial resolution over a field of view of $450\mu \mathrm{m}$ . The confocal microendoscope is able to focus down to a depth of $200\mu \mathrm{m}$ in the tissue. For the experimental results shown thereafter, the depth of observation is not precisely known. Images were acquired by focusing on the tissue surface. The optical power incident on the surface of the tissue is typically less than $1\mathrm{mW}$ . This yields a power density over the field of view of roughly $630\mathrm{mW}∕{\mathrm{cm}}^2$ .

The camera used in the multispectral confocal microendoscope is a Photometrics Cascade II CCD (Photometrics, Tucson, Arizona) with a $512\times 512$ detector matrix. Each detector element is $16\times 16\mu \mathrm{m}$ . The camera is capable of recording 30 images per second, so a 3-D dataset consisting of 150 spectral samples is collected in about $6\mathrm{s}$ . Methods to increase the collection speed are presented in Sec. 4.

Spectral resolution is defined as the minimum resolvable wavelength difference. In our system, the theoretical spectral linewidth is determined by the convolution of the geometrical image of the slit on the detector, the camera pixel function, and the PSF of the camera lens. The theoretical FWHM of the spectral linewidth is $1.9\text{pixels}$ and does not change significantly with wavelength within our spectral bandwidth. The spacing between two resolvable wavelengths is then approximately 2 camera pixels. This spacing can be correlated to a change in wavelength using the wavelength calibration procedure described. Because refractive index is a nonlinear function of wavelength, the dispersion of light through the prism is not constant. Consequently, the spectral resolution is not constant with wavelength. Our system achieves a spectral resolution of $2.9\mathrm{nm}$ at a wavelength of $500\mathrm{nm}$ and $8.4\mathrm{nm}$ at a wavelength of $750\mathrm{nm}$ . Table 1 shows the spectral resolution with respect to wavelength at intervals of $50\mathrm{nm}$ . At this stage of the system development, the recorded spectra are not corrected for the camera spectral sensitivity.

Table 1

Wavelength-dependent spectral resolution.

3.2.

Results from Grayscale Imaging

The grayscale imaging mode of the confocal microendoscope has been used to investigate a wide variety of human and animal tissues. The results presented here were obtained through topical administration of acridine orange (AO) (Invitrogen, Eugene, Oregon), which is a vital dye that stains DNA and RNA. Figure 6 shows an ensemble of grayscale confocal microendoscope images obtained from excised human tissues and in a mouse¹¹ that was sacrificed immediately prior to an in situ experiment. For each biological sample, about $10\mu \mathrm{L}$ of AO ( $0.1\mathrm{mg}∕\mathrm{mL}$ of concentration) was applied to the tissue surface. Imaging was started almost immediately.

Fig. 6

Grayscale results from excised human tissues and in situ animal tissues. (a) Normal appearing epithelium of human ovary, (b) unusual area of human ovary, (c) normal tissue of human esophagus, (d) human Barrett’s esophagus, (e) peritoneal wall of a mouse, and (f) liver of a mouse.

Figures 6a and 6b show results obtained from excised human ovary. One can clearly visualize significant differences between the two images. Figure 6a shows a uniform layer of cells, which is typical for healthy ovarian epithelium. Figure 6b shows an abnormal pattern of cells that is indicative of pathology. These data were histologically verified as normal for Fig. 6a and cancerous for Fig. 6b.

Figures 6c and 6d show images obtained from excised human esophagus. The sample in Fig. 6c was diagnosed as normal squamous epithelium, while the sample in Fig. 6d was diagnosed as Barrett’s esophagus, which is a common precursor to esophageal adenocarcinoma. The sparse punctate nuclei of Fig. 6c and the glandular structure of Fig. 6d are in agreement with the histological findings. The images presented in Figs. 6a, 6b, 6c, 6d were collected over several experiments and as such, the absolute brightness and contrast between images should not be compared.

Figures 6e and 6f show images of mouse peritoneal wall and liver, respectively. The animal used in this experiment was considered healthy, and no specific abnormalities were observed in the images.

3.3.

Results from Multispectral Imaging

Preliminary multispectral imaging experiments were performed using a phantom containing a mixture of $15\text{-}\mu \mathrm{m}$ -diam microspheres (Invitrogen, Eugene, OR) with different spectral properties. Figure 7 shows images of these microspheres at two different spectral planes of the multispectral data cube.

Fig. 7

Fluorescence images from $15\text{-}\mu \mathrm{m}$ microspheres at wavelength (a) $578\mathrm{nm}$ and (b) $639\mathrm{nm}$ , and spectra associated with image point indicated by the white cross.

Some beads are visible at $639\mathrm{nm}$ (a) that are not visible at $578\mathrm{nm}$ (b). The difference in the spectra shows that two groups of microspheres are present in the sample. The first group has a peak emission around $640\mathrm{nm}$ , while the second group has a peak emission at $600\mathrm{nm}$ .

Figure 8 shows multispectral data obtained from an excised human esophagus stained with AO. This biological sample was from a patient diagnosed with Barrett’s esophagus. Several drops of AO ( $0.1\mathrm{mg}∕\mathrm{mL}$ of concentration) were topically applied to the tissue. Imaging was initiated almost immediately and continued for roughly $20\min$ . The left image shows a low wavelength emission at $540\mathrm{nm}$ , while the right image shows a high wavelength emission at $670\mathrm{nm}$ . Different structured features are observed at the different wavelengths. This is also visible on the side windows (representing one spatial dimension as a function of wavelength), where the intensity distribution varies depending on the wavelength and exhibits two broad spectral regions of fluorescence. The spectrum at the bottom of each image is plotted for the point in the image indicated by the white cross. The presence of two distinct spectral structures is confirmed by the dual peaked spectra. Although the tissue is from a patient with Barrett’s esophagus, typical glandular structures of Barrett’s [see Fig. 6d] were not observed for this sample (not even in a slide made from this tissue and viewed with a standard microscope), and a normal-looking distribution of squamous epithelium corresponding to nuclei are visible at the low wavelength. At the high wavelength, a uniformly distributed bright region is caused by the fluorescence from cellular cytoplasm. Nuclei are still visible in the background. Some dark areas correspond to the voids left by blood vessels that were washed off in the process of biopsy. AO typically shows green fluorescence in the nuclei and red fluorescence in the cytoplasm when excited at $488\mathrm{nm}$ . However, the peak and linewidth of AO’s fluorescence spectrum varies depending on the pH in the tissue and the concentrations of the solvent and other cellular structures.^{16, 17, 18}

Fig. 8

Human Barrett’s esophagus tissue stained with AO (a) at a low wavelength and (b) at a higher wavelength. The spectrum associated with the image point indicated by the white cross is displayed below each image.

Figure 9 shows images of the peritoneal wall of a mouse stained with AO and obtained in situ. The mouse was sacrificed prior to the experiment and opened up to allow insertion of the catheter. A few drops of AO ( $0.1\mathrm{mg}∕\mathrm{mL}$ of concentration) was deposited on the peritoneal wall. We started imaging about one minute later and continued for approximately $10\min$ total. Again, some structures are visible at the higher wavelength $\left(762\mathrm{nm}\right)$ , but not at the lower wavelength $\left(572\mathrm{nm}\right)$ . These organelles might be lysosomes. The spectra associated with the cursor location are also shown; the spectrum on the left has a peak around $570\mathrm{nm}$ , while the spectrum on the right, corresponding to the organelles, exhibits a peak around $740\mathrm{nm}$ . Thus different aspects of the tissues are distinguishable through their emission spectra.

Fig. 9

In situ images of the peritoneal wall of a mouse stained with AO at different wavelengths, (a) $572\mathrm{nm}$ and (b) $762\mathrm{nm}$ . The spectrum associated with the image point indicated by the white cross is displayed below each image.

Figure 10 shows data obtained from a rat intestine after tagging it with DiA. DiA is a fluorophore that is used to label cytoplasmic membranes, with a peak emission at $586\mathrm{nm}$ and a peak excitation at $454\mathrm{nm}$ (Invitrogen, Eugene, Oregon). Tissues were topically stained with DiA at a concentration of $2.5\mathrm{mg}∕\mathrm{mL}$ . Different structures are observed when scanning from low to high wavelengths (from left to right in Fig. 10). The spectra associated with these structures are plotted below the corresponding images and also confirm the presence of different staining patterns. The left spectrum has a broad peak in the 610-to-640-nm range, while the spectrum in the other two images shows a broad peak in the 660-to-710-nm range.

Fig. 10

Rat intestine stained with DiA at a (a) low wavelength, (b) intermediate wavelength, and (c) high wavelength.

The images demonstrate that the multispectral confocal microendoscope can differentiate different tissue types and provide high quality images across the whole field of view. Moreover, the automated data acquisition system and processing software that has been developed yield spectral information that has the potential to provide additional in vivo diagnostic information. As a consequence, distinct elements of a given tissue sample can be highlighted based on spectral differences in their fluorescence properties.