We present the design and feasibility testing of a multimodal co-registered endoscope based on a dual-path optical system integrated with a scanning piezo. This endoscope incorporates three different imaging modalities. A large field-of-view (FOV) reflectance imaging system enables visualization of objects several millimeters in front of the endoscope, while optical coherence microscopy (OCM) and multiphoton microscopy (MPM) are employed in contact with tissue to further analyze suspicious areas. The optical system allows multiple different imaging modalities by employing a dual optical path. One path features a low numerical aperture (NA) and wide FOV to allow reflectance imaging of distant objects. The other path features a high NA and short working distance to allow microscopy techniques such as OCM and MPM. Images of test targets were obtained with each imaging modality to verify and characterize the imaging capabilities of the endoscope. The reflectance modality was demonstrated with a 561 nm laser to allow high contrast with blood vessels. It achieved a lateral resolution of 24.8 μm at 5 mm and a working distance from 5 to 30 mm. OCM was performed with a 1300 nm super-luminescent diode since this wavelength experiences low relative scattering to allow for deeper tissue imaging. Measured OCM lateral and axial resolution was 4.0 and 14.2 μm, respectively. MPM was performed with a custom 1400 nm femtosecond fiber laser, a wavelength suitable for exciting multiple exogenous, and some endogenous fluorophores, as well as providing information on tissue composition through harmonic generation processes. A 4.0 μm MPM lateral resolution was measured. |
1.IntroductionIt is estimated that cancer affects about 450 per 100,000 people in the United States alone.1 Cancer detection at early stages is critical to a patient’s outcome since cancer is localized and is most responsive to treatment at the early stages. Unfortunately, detecting localized precancerous lesions is difficult due to the lack of symptoms and the microscopic size of early neoplastic lesions. Making a definitive diagnosis with histology requires a destructive surgical biopsy that can miss the affected tissue area. Therefore, a faster and less invasive detection method that can survey more tissue is desired to make the process efficient and accurate. Optical imaging has been proven to detect several types of early cancer effectively. Furthermore, multimodal optical imaging may provide improved classification accuracy.2 Research teams, including ours, have been developing novel endoscopes that feature dual-modality imaging to reach internal organs.3–6 A multimodal endoscope might detect precancerous lesions and reduce unnecessary biopsies. 1.1.Optical Imaging MethodsOptical imaging techniques including reflectance and fluorescence imaging, confocal microscopy, multiphoton microscopy (MPM), and optical coherence tomography (OCT) have proven to be sensitive and specific methods of detection for different cancers.7–18 A wide-field imaging technique, such as reflectance or fluorescence imaging, can first be used to identify suspicious areas of tissue that may contain cancer. These areas can then be further investigated using a high-resolution technique such as MPM, OCT, or optical coherence microscopy (OCM). Our endoscope combines the three modalities: wide-field narrow-band reflectance imaging, MPM, and OCM. Reflectance imaging obtains contrast from tissue scatterers and absorbers, particularly hemoglobin. Narrow-band illumination in the blue or green wavelength region enables the identification of regions with increased vascularity, a hallmark of cancer.19 OCM provides depth-resolved, micron-scale resolution images of tissue microstructure using reflected near-infrared (NIR) light. OCM uses a high numerical aperture (NA) objective to achieve high lateral resolution while sacrificing the extended depth of focus typically achieved with OCT. Two multiphoton processes of interest in MPM are two-photon excited fluorescence (2PEF) and second harmonic generation (SHG). 2PEF occurs when multiple photons are absorbed nearly simultaneously by a molecule. This molecule then emits a single photon at a wavelength shorter than the incident light yet longer than half the incident wavelength. Fluorescent molecules, including some with excitation or emission in the NIR such as lipopigments and porphyrins, are known to be altered in cancerous tissue. SHG is generated by non-centrosymmetric molecules, such as collagen, which emit light at exactly half the excitation light wavelength. Therefore, combining reflectance, OCM, and MPM in an endoscope may provide a highly sensitive and specific tool for the in vivo detection of cancer in its early stages. As such, it could serve as a minimally invasive alternative to surgical biopsy and histology. Additionally, the co-registered imaging of known features of cancerous tissue, such as increased vascularity, modified tissue and cellular microstructure, altered collagen structure, and changes in biomolecule concentration, can potentially provide additional information about the health of the tissue, further increasing the sensitivity and specificity over a single-modality system. 1.2.Goals of this StudyWith the intent of providing a tool to detect abnormal cells and tissue indicative of early stage cancer, we designed a rigid endoscope that can perform co-registered visible light reflectance imaging for navigation, and OCM and MPM for detailed tissue interrogation. For operational functionality, two additional channels are included in the endoscope, one for irrigation and another for biopsy forceps or other small-diameter tool use. This triple-modality endoscope is small enough for minimally invasive imaging and could be used as an optical screening method for various cancers in the future. For example, it could be utilized as a transvaginal salpingoscope to screen women at high risk for ovarian cancer. It could also be used during surgical laparoscopy to locate cancer margins or metastases, similar to previous studies.20,21 The endoscope is designed so that the narrow-band, wide-field navigation modality could be used to safely insert the device and navigate into the peritoneal cavity and identify suspicious areas. Once a suspicious area is identified, the front face of the endoscope could be placed in contact with the tissue to perform high-resolution imaging with OCM or MPM. The goal of this study is to describe our endoscope and demonstrate proof of concept. 2.Requirements2.1.Functional and Physical RequirementsIn cooperation with our physician collaborators, a set of functional and physical requirements were derived for the endoscope intended to be used primarily as a transvaginal salpingoscope. These requirements are summarized in Table 1. The endoscope requires a forward-viewing reflected-light modality with a wide field of view (FOV) to enable insertion, navigation to the tissue of interest, and rapid identification of abnormalities. Tissue suspected to be abnormal would then be imaged at high resolution with OCM and MPM to confirm findings. Ideally, the navigation and high-resolution modalities should be co-registered. Additional requirements for clinical utility include an irrigation channel that can either deliver saline for cleaning the optics at the distal tip of the endoscope or deliver exogenous contrast agents for improved labeling of pathological tissue. Additionally, an accessory channel for biopsy tools enables the collection of a small tissue sample for conventional methods of diagnosis. Table 1Requirements for the endoscope.
The ability to reach organs of interest, such as the ovary, dictates the physical requirements for both the insertable distal section and the non-insertable proximal section (handle). For the insertable portion, we established basic requirements for rigidity, diameter, length, biocompatibility, and sterilizability. We chose to make the distal section completely rigid for familiarity and allow the physician to have an explicit reference for the endoscope’s location. Therapeutic endoscopes with diameters as large as 7 mm are often utilized. However, for diagnostic purposes, diameters of are preferred to minimize tissue trauma. This diameter is large enough to encapsulate the optics and the irrigation and biopsy accessory channels. The length of the insertable section should be large enough for the tip to reach the tissues of interest in a variety of patient anatomies. It was estimated that an insertable length of would be sufficient for imaging of ovaries and fallopian tubes. All exposed components must be made from biocompatible materials and be sterilizable without losing performance. 2.2.Optical RequirementsClinically relevant requirements for FOV, working distance, resolution, imaging speed, illumination wavelength, and power were established for all of the optical modalities. A summary of the optical requirements is presented in Table 2. Table 2Summary of optical requirements.
2.3.Reflected Light Modality (Navigation) Optical RequirementsThe FOV for the navigation modality should be sufficient to identify the target tissue and avoid damage to adjacent tissues, ideally 60 deg full FOV, with 25 deg considered adequate. Working distance should be at least 5 to 20 mm, with a lateral resolution of better than at 5 mm distance at the center of the field. These requirements allow adequate navigation to identify focal areas of disease that are abnormal clusters of 10 or more cells.22 Video-rate (i.e., ) imaging speed is required to reduce motion artifacts and to enable rapid identification of abnormal areas, so that most of the allowable screening time can be spent with high-resolution modalities. The illumination source should provide high contrast for navigation purposes. For monochrome imaging, blue to green wavelengths are preferred to enhance blood contrast. Laser safety standards should comply as per ANSI Z136.1 standard. 2.4.High-Resolution Modalities Optical RequirementsThe FOV of the high-resolution modalities should be sufficient to image a representative amount of tissue (e.g., at least 10 cell diameters or about ). The high-resolution mode is utilized when the endoscope is near or in contact with the tissue. A focal distance of enables visualization of simple epithelia and acceptable tissue attenuation. Cellular resolution ( and axial ) is necessary to identify pathology. Although having the endoscope in contact with tissue mitigates motion artifact, rapid imaging, ideally at video rate, is still desired. This rapid imaging allows the simultaneous SHG and 2PEF imaging modalities to image over , and OCM to image over 3 cm linear within the allowable screening time. For OCM and MPM imaging modalities, illumination wavelengths in the NIR range are preferred due to relatively low attenuation and significant penetration for tissue imaging. An OCT source with a 1300 nm center wavelength is ideal for tissue penetration and availability of photonic components. A bandwidth of at least 75 nm is desired to achieve axial resolution in tissue. A variety of wavelengths of MPM lasers are useful, with wavelengths between 1200 and 1500 nm providing advantages of low tissue attenuation as well as ability to excite NIR endogenous fluorophores or exogenous dyes. The MPM source must also provide short pulses with durations in the 100 to 200 fs range to achieve high-excitation efficiency but minimize non-linear processes in fiber while maintaining a high repetition rate for rapid image acquisition and should comply with the ANSI Z136.1 standard. 2.5.Data Acquisition RequirementsMPM and OCM are techniques in which the image is reconstructed by scanning the FOV point by point or column by column, respectively. Navigation imaging could be performed with either point-by-point reconstruction or a two-dimensional snapshot. However, the former technique was chosen to maintain compatibility with MPM and OCM. As such, a distal two-dimensional scanning system is necessary. Given the need for fiber deliver of light through the endoscope, we have chosen to use a quartered piezo scanning system with cantilevered fiber, described in detail later. A sufficiently fast detector and data acquisition (DAQ) method must be implemented to meet imaging sampling and speed requirements. Video-rate imaging with per image in the MPM and reflectance imaging modalities requires a sensor with a bandwidth of 1 MHz or greater. Variable gain is useful to account for differing signal strengths. Photomultiplier tubes (PMT) and associated amplifiers used for detecting MPM signals must be highly sensitive in the red to NIR wavelengths and have gain-bandwidth products sufficient for handling weak high-speed signals. Preferably, both MPM signals should be detected and digitized synchronously to ensure co-registered imaging. Since we implemented spectral domain configuration for OCM, the associated line scan camera must be capable of readout speeds in the kHz to obtain video-rate cross-sectional OCM images with 100 line scans or more (i.e., columns) per image. An equipment rack should enclose all sources, detectors, and computational equipment. It should also be portable and comply with electrical and laser safety requirements. The software interface should control imaging parameters and display images to the user. Table 3 summarizes the acquisition requirements. Table 3Summary of acquisition requirements.
3.Multimodal System DesignFigure 1 shows a diagram for the multimodal endoscope system designed and built to meet the requirements in Sec. 2. The system can be divided into the handheld endoscope and the instrument rack. The endoscope consists of the insertable portion, the handle, and the long flexible tubing relaying light and power between the handle and rack, while the instrument rack contains of the optical and photonic components as well as the computer. Fig. 1Endoscope system diagram. Green outlined components are for navigation. Red outlined components are for MPM imaging. Brown outlined components are for OCM imaging. Components outlined in blue are shared by two or more modalities. Various fiber types, electrical wire, and tubing are color coded. ![]() 4.Endoscope DesignA photograph of the endoscope is shown in Fig. 2. The insertable portion consists of a rigid hypotube protecting the fibers, electrical wires, and accessory channel tubing and contains a distal ferrule that holds the imaging optics, the quartered piezo scanning system, light collection fibers, and tubing for the irrigation and accessory channels. The hypotube is made of medical-grade stainless steel (304H09XX, MicroGroup (Medway), Massachusetts, USA), custom-cut to set the insertable section’s length at 30 cm with an outer diameter of 3.733 mm and a wall thickness of . The non-insertable ergonomic handle is 3D-printed from rigid VeroBlack (Stratasys, Minnesota, USA) material. It contains the irrigation and accessory ports as well as channels for the fibers and piezo control wires. 4.1.Distal Tip ConfigurationA diagram and photograph of the components housed within the distal ferrule (no protective hypotube), and a distal tip end view of the completed endoscope, are shown in Figs. 3 and 4, respectively. Fig. 3(a) Rendering and (b) photograph of the endoscope distal assembly under construction. The protective hypotubing is not yet installed. ![]() The custom ferrule (Majer Precision Engineering, Inc., Arizona, USA) is made of medical-grade stainless steel and was designed to hold all the optical components, the scanning system, and the irrigation and accessory channel tubing in a rigid configuration. It has a small outer diameter of 3.3 mm, which fits inside the hypotubing of the insertable portion. The ferrule’s complex shape was achieved using fine wire electrical discharge machining (EDM). Within the ferrule, the largest cutout (1.52 mm diameter) holds the quartered piezo scanning system and a lens system, used for illumination in all three imaging modalities (navigation, OCM, and MPM) and collection of remitted light for OCM and MPM. Twelve small cutouts hold collection fibers for the remitted navigation light. A 1.35 mm diameter cutout holds the polyimide accessory channel tubing (480-II, Microlumen, Florida, USA), which has a 1.34 mm outer diameter and 0.06 mm wall thickness. The resulting 1.22 mm clear channel is sufficiently large to pass accessories such as biopsy forceps (SpyBite, M00546270, Boston Scientific Corporation, Massachusetts, USA). The final 0.97 mm diameter cutout holds smaller diameter tubing (335-II, Microlumen, Florida, USA) with an outer diameter of 0.96 and a 0.05 mm wall thickness. This tubing is used for irrigation with saline and the delivery of contrast agents. The latter two functionalities are shown in Fig. 5. 4.2.Optical and Scanning System ConfigurationAll three optical modalities utilize a common double-clad fiber (DCF) (DCF13, Thorlabs, New Jersey, USA), scanning system, and lens system. All modalities utilize the single-mode core of the DCF for illumination. Based on the requirements in Sec. 2, the specified illumination wavelength, remitted signal wavelength, illumination spot size (resolution), illumination FOV, and collection mechanism for the three modalities are specified in Table 4. Table 4Illumination and collection mechanisms for the different optical modalities.
An illumination wavelength of 561 nm was chosen for navigation based on laser availability and optical tissue properties. This wavelength provides high contrast when imaging blood vessels due to the oxy- and deoxy-hemoglobin absorption peaks near this wavelength.23 A laser diode with 35 mW power output was used (R560503FX, LaserGlow Technologies, Toronto, Canada). For OCM, a dual-superluminescent diode with a center wavelength of 1300 nm, 70 nm full-width-half-maximum bandwidth, and 8 mW power was chosen (Broadlighter 1300, SuperLum, Co., Cork, Ireland). For MPM, a custom , 40 MHz pulsed laser centered at 1400 nm with 160 mW output power was utilized.24 Laser pulse dispersion in the optical fiber was mitigated by prechirping with a grating pair. The power output measurements at the distal tip of the endoscope were lower than the source outputs, and they were confirmed to meet ANSI Z136.1 standard for permissible laser exposure on skin. The beam rapidly diverges from the endoscope and operational controls (e.g., lasers only energized with endoscope inside the patient) limiting eye safety concerns. The differing illumination spot sizes and FOVs of the reflectance system compared to those of the OCM/MPM system are made possible by the novel lens system previously described.25 In brief, the lens system contains a triplet with two dichroic coatings at the distal and proximal inner surfaces in the shape of a dot and a ring, respectively. The coatings are arranged similar to the primary and secondary mirrors of a telescope. The coatings transmit light at visible wavelengths so that the final objective lens weakly focuses the light into a low 0.007 NA beam with a depth of focus from 5 to 22 mm and a central spot size of at the best focus. The reflectance signal from the illuminated spot is collected with twelve 0.5 NA multimode fibers held by the ferrule, enabling good light collection efficiency. At NIR wavelengths, the coatings reflect light in a telescope-like configuration to expand the beam, and the final objective lens strongly focuses them to a high 0.44 NA beam with a front focal distance and an spot size. The OCM light is collected back into the single-mode core of the DCF, as required for this interferometric technique. The MPM signal is collected in the 0.2 NA first cladding of the DCF, which enables increased light collection over utilizing the core only. Note that MPM only captures the central portion of the collected light since the wavelengths generated by the non-linear effects are not affected by the dichroic surfaces. The optical paths used are illustrated in Fig. 6. Fig. 6Tunnel diagrams, where light propagates form left to right including reflected light, for the illumination, optical paths, and collection mechanisms for the different optical modalities. The sample is represented with an orange line. ![]() The quartered-piezo (piezo) scanning assembly with a cantilevered DCF and fiber mass is shown in Fig. 7. Scanning endoscopes have been demonstrated before using Lissajous and spiral scanning patterns.26 Our endoscope works with a raster pattern since relatively slow (tens of Hz) scanning is required for OCM, and relatively fast (thousands of Hz) scanning is required for video-rate en face navigation and MPM imaging. Raster scanning can also ease image reconstruction because it is acquired in a more intuitive sequence than spiral or Lissajous scanning. However, image calibration adjustment is still needed due to the sinusoidal motion of the piezo. Raster scanning at slow and fast frequencies can be achieved by utilizing multiple vibrational resonance frequencies (e.g., first and third) that can be adjusted by tuning the fiber length and utilizing fiber-mounted masses.24 Fig. 7Quartered piezo scanning assembly including the quartered piezo tube, fiber, wires, collar, and brass mass. The brass mass is shown sectioned through the middle to show the internal “bow-tie” shape, the fiber channel, and the hole used to glue the mass to the fiber. ![]() The scanning system shown in Fig. 7 utilizes a custom piezo with 1.0 mm outer diameter, 0.2 mm wall thickness, 11 mm length, 4 longitudinal outer electrodes (electrically isolated quadrants at 90 deg), and PIC255 material (PI, Massachusetts, USA). A custom polyether ether ketone piezo collar (Optics Technology, New York, USA) holds the proximal piezo stationary and centered inside the ferrule cutout while allowing the distal piezo end and the cantilevered fiber to oscillate with relative freedom. Additionally, the collar protects the exposed leads that apply voltage to the piezo electrodes. Desired resonant frequencies of first resonance and third resonance were achieved using 16 mm long cantilevered fiber with a custom brass mass (Optics Technology). The mass shape is cylindrical to maximize its weight in the 1.52 mm ferrule cutout while allowing a tip deflection amplitude of up to peak-to-peak. The cylinder’s inner diameter was shaped in a “bow-tie” fashion and glued to the fiber at one point only, through a hole on the side of the mass. Thus the fiber is free to bend and oscillate as required at third resonance. The mass is placed such that the center of mass sits at a location between 75% and 80% of the fiber’s length, aiming for the third resonance distal nodal point. The mass, placed as described, lowers the first harmonic frequency while keeping the third harmonic frequency relatively unchanged,24 allowing for a 100× difference in the two frequencies for raster scanning. The ratio of scanning frequencies limits the number of pixels in one dimension of the image (the slow axis), in this case, to 100 pixels, and the DAQ board sampling rate of 2 MHz limits the number of pixels in the other dimension (the fast axis) to about 130 pixels. For the high NA imaging modalities, 100 pixels enable Nyquist sampling given the FOV and resolution. For low NA reflectance imaging, the image may be undersampled depending upon the FOV chosen. However, it is worth noting that the number of image pixels obtained is similar to a 10,000 elements fiber bundle, which has been widely used for miniature endoscopes.27 4.3.Endoscope HandleThe fibers, piezo control wires, and irrigation and accessory tubing are carried through the hypotube to the 3D-printed handle, where they are separated into three different channels, as shown in Fig. 8. The center channel contains the 12 multimode collection fibers, the dual-clad fiber, and the four piezo control wires. A curved side-channel houses the accessory channel tubing with a 1.22 mm inner diameter. The tapered entrance allows the tool to be inserted smoothly. A second side-channel holds the Luer lock tip of a 21G hypodermic needle with a shortened cannula (Exelint International, California, USA), which is mated with and glued into the irrigation channel tubing. This Luer lock connector provides an attachment point that allows for the introduction of saline or contrast agents with a standard Luer lock syringe. Fig. 8(a) Open handle showing the internal channels guiding the different components and (b) handle with components installed. ![]() Proximal to the handle, the fibers and piezo control wires are protected with a 2.0 m long, 3/16 in. outer diameter vinyl tube (4027470, ACE, USA). The tubing is glued to the handle and protected with a separate heat shrink tubing at the handle interface (MT5000-1/2-0-SP, TE Connectivity, Schaffhausen, Switzerland). Components exit the vinyl tube’s proximal end and lead into three connectors that attach the handheld endoscope to the instrument rack. A standard FC/APC connector is used for the DCF, a standard SMA connector is used for all 12 multimode fibers combined into a single lumen, and a 4-pin connector is used for the piezo control wires (SP13-4pin, Szjelen, China). A complete prototype of the handheld endoscope is shown in Fig. 9. All exposed materials were carefully selected to ensure compatibility and sterilizability of the endoscope. The hypotube is made of 304 stainless steel, and a custom cover glass is made of fused silica (W-SIO2UVE-RS0003, Bern Optics, Massachusetts, USA). The irrigation and accessory tubing is made of polyimide. The gaps between the cover glass and the hypotube and the gaps between the inner irrigation and accessory tubing and the cover glass are sealed with a medical-grade epoxy (EP42HT-2MED Clear, Masterbond, New Jersey, USA). Gaps between components such as the handle-hypotube interface are further sealed with medical-grade adhesive (Loctite 4013, Henkel Adhesives, Düsseldorf, Germany). The 3D-printed handle material (VeroBlack) and the vinyl tubing are not biocompatible, but they can be covered with a sterile ultrasound sheath during clinical use. Sterilizability of the entire handheld endoscope was confirmed with deliberate contamination, then swabs for common pathogens after hydrogen peroxide sterilization. All tests revealed that no pathogens were present after sterilization. 5.Instrumentation Rack SystemsThe rack contains all source, detector, and computational components needed to operate the endoscope.28 The rack contains an electrical power isolator (IS1800HG, Tripp Lite, Illinois, USA) and conforms to leakage current safety requirements [IEC 60601-1 part 1, third edition, part 1, section 8.7.3d]. 5.1.Navigation ComponentsReferring to the system diagram in Fig. 1, the 561 nm laser is connected to a custom wavelength division multiplexer (WDM) (OZ Optics, Ontario, Canada) optimized for combining this wavelength with the 1300 nm OCM source. The WDM output then enters a custom WDM/dual-clad fiber coupler (DCFC) (DBM-1300-200901, Castor, Quebec, Canada), which adds in the 1400 nm MPM source and directs all wavelengths into the core of the DCF that runs through the endoscope. The single lumen SMA connector containing light reflected from the sample is mated to an SMA connector containing a single core fiber at the instrument rack interface (M93L01, Thorlabs). This fiber is coupled to an SMA-adapted silicon detector (New Focus 2107-FC-M, MKS, California, USA) with a 6-MHz bandwidth at the gain used during testing. 5.2.OCM ComponentsThe 1300 nm OCM source is coupled into a fiber interferometer containing a circulator (37474 FOC-12N-11, OZ Optics) and a 90:10 fiber coupler (FUSC-280125223, Precision Micro-optics, Massachusetts, USA). Ten percent of the power is directed into a reference arm with dispersion matching, polarization control, and an optical path length adjustable to match the endoscope. The remaining 90% of the power is directed into the WDMs for passage through the DCFC and output through the DCF core. Light reflected by the tissue into the DCF core follows the same return path, where it is recombined with reference arm light and directed by the circulator to the spectrometer. The spectrometer is equipped with a reflective collimator (MPD124-M01, Thorlabs), 300 mm focal length custom-made lens (AC254-300-C-ML, Thorlabs), reflective grating (GR25-0613, Thorlabs), and a 2048 pixel linear array (GL2048L, Sensors Unlimited, New Jersey, USA). This camera was chosen due to its high quantum efficiency (60% to 70%), user-controlled exposure, high dynamic range (), and appropriate read rate of up to 76,000 lines per second. 5.3.MPM ComponentsMPM source light from the custom 1400 nm femtosecond laser is combined with the other wavelengths into the DCF core via the second WDM and DCFC. Remitted light in the DCF inner cladding is delivered via the multimode arm of the DCF to a custom-made PMT enclosure. A reflective collimator (RC02FC-P01, Thorlabs) delivers the light to a series of dichroic mirrors that separate SHG and 2PEF (FF01-890/SP, FF725-SDi01-, FF01-850/310-25, FF01-770/SP-25, Semrock, New York, USA) and direct the signals to PMTs (H10721-20, Hamamatsu, Iwata, Japan). Transimpedance amplifiers (PMT-4V3, Advanced Research Instruments Corp., Oregon, USA) provide variable gain with a 2 MHz bandwidth at the gain used in testing. A programmable filter (3945, Krohn-Hite, Massachusetts, USA) can be used to remove high-frequency noise. 5.4.Data Acquisition, Software, and DisplayTo generate piezo control signals, acquire detector/camera data, and reconstruct and display images, a custom LabView program was written in collaboration with INTAQ (Arizona, USA). Based on user input image dimensions and image rate, piezo control signals for each axis are generated and output through a multipurpose DAQ board (PCIe-6321, National Instruments, Texas, USA). The control signals are further amplified at the piezo driver (TD250, Piezo Drive, Shortland, Australia). One-dimensional slow-axis scanning is used for OCM imaging, and two-dimensional fast- and slow-axis raster scanning is used for reflectance and MPM modes. Piezo control signals are synchronized with the data acquisition. Reflectance and MPM signals are acquired with the same multipurpose DAQ board, capable of up to four synchronous analog input channels at up to 3.5 MHz each (PCIe-6374, National Instruments). Due to limitations in the current software only acquiring on one side of the sinusoidal piezo movement, the number of pixels in the slow scanning imaging dimension is limited to 50. The number of pixels in the fast scanning dimension is only limited by the sampling rate of the DAQ board and the piezo fast scan frequency and could be up to 460. OCM line scan camera signals are acquired with a frame grabber (PCIe-1433, National Instruments). The number of A-scans (line scan camera reads) per image is limited by the camera line rate and the piezo slow scan frequency and could be up to 950. The software enables the acquisition and the display of navigation, OCM, or MPM images. Navigation and MPM images are displayed in real-time in their as-acquired form, which is distorted due to the scanning system’s sinusoidal movement, piezo scanning axes that may not be perfectly perpendicular, and minor stable whirling of the fiber. In practice, typically 10% of data points at each image edge are excluded, as the fiber movement at these locations is so slow as to cause significant distortion and provide limited additional FOV. Therefore, the number of pixels in the slow scanning imaging dimension is normally 40. A calibration obtained from imaging a grid pattern can be applied to process images into their true dimensions. OCM camera reads are processed by applying calibration files created using published methods29 and taking the Fourier transform of each A-scan. B-scan images are constructed by combining processed A-scans. Acquired raw data, processed data, and configuration files are saved upon request. 6.Results6.1.Reflectance TestThe endoscopic optical system was tested in a tabletop configuration without the hypotube to allow for alignment of the components. Reflectance modality images of a black on white paper USAF resolution target were analyzed (#53-715, Edmund Optics, New Jersey, USA) to evaluate FOV and lateral resolution. At a working distance of 5.0 mm, up to a FOV could be imaged, which equates to a 28 deg FOV. To evaluate lateral resolution, the FOV was reduced to by controlling the piezo oscillation amplitude voltage. The reduction of the FOV resulted in distance between pixels when a square image of was obtained. The contrast was calculated utilizing the grayscale values of the images, and the resolving criterion was set to a contrast of 0.01 or greater using: Grayscale values were read from a line traced through the horizontal or vertical bars of the respective element. An or was taken at each line’s inflection points, and four contrast values were calculated, one for each adjacent and values, using the contrast equation. The average of these contrast values is the reported value. Figure 10 shows reflectance images of the USAF resolution target. The smallest element with contrast above the threshold is group 4 element 3, equivalent to a lateral resolution of or 20.16 lines per millimeter. Vertical lines displayed a contrast of , and horizontal lines displayed a contrast of . The differences in contrast between the horizontal and vertical fringes seen in Fig. 10 may be attributable to the suboptimal alignment of the lenses causing astigmatism. Fig. 10Reflectance images of a USAF target showing the (a) vertical and (b) horizontal lines for group 4 element 3. The contrast and brightness of these images were digitally adjusted for clarity. (c) Grayscale values extracted from the red lines traced over the horizontal and vertical patterns shown on the left side. ![]() To further test the capability of reflectance imaging and to measure the working distance range, images of a reflectance target were obtained at 5, 10, 15, and 30 mm from the endoscope distal cover glass. The target consists of a surface coated with Spectralon (Labsphere Inc., New Hampshire, USA) and geometrical shapes made of Fineshut KIWAMI (Koyo Orient Japan Co. Ltd., Saitama, Japan). These materials provide some of the highest and lowest, respectively, reflectance values obtainable, thus providing higher contrast than the USAF target. The reflectance images and the target white light image can be seen in Fig. 11. The finite thickness of the Fineshut material causes some artifacts, especially at close range. The reflectance images show that the shapes are visualized over the entire range with a contrast of up to 0.58. At 30 mm, the low collection efficiency of the reflected signal results in a noisier image. 6.2.OCM TestTo test OCM, the lateral FOV was reduced to by controlling the piezo oscillation amplitude voltage. The reduction of the FOV resulted in distance between A-scans with a total of 166 lines per frame. The best focus was found to be at from the distal cover glass. A variable line grating test target was imaged to test OCM lateral resolution (R1L3S6P, Thorlabs). The OCM image is shown in Fig. 12. This test was performed as a simple pass/fail test for OCM since the system’s lateral resolution is theoretically , but the smallest pattern in the target is . This image shows excellent contrast between the lines in the lateral direction with a signal-to-background ratio (SBR) of about 68. The FWHM of the line in the axial direction (vertical on the image) is significantly degraded from the theoretical axial resolution of . We attribute the discrepancy to suboptimal calibration of the spectrometer. 6.3.MPM TestThe broken edge of a gallium arsenide (GaAs) sample was imaged to test the MPM modality due to the strong emission of 2PEF at the edge. The FOV was reduced to by controlling the piezo oscillation amplitude voltage. The reduction of the FOV resulted in distance between pixels when a square image of was obtained. The best focus was found to be at from the distal cover glass. The multiphoton image obtained by averaging 32 frames is shown in Fig. 13. The edge image allows us to use a non-linear technique to characterize lateral resolution.30 This technique consists of fitting the edge profile using: where is lateral displacement of the scanning sampling, and , , and are the fitting coefficients. The resolution of the system is given by multiplying the fitting coefficient by 1.17. A simple MATLAB code was generated to perform this task.The resolution computed by this technique is . Additionally, an image of a computer chip was acquired to further test the MPM capability and determine SBR. The image can be seen in Fig. 14, showing two wires that are used to connect the chip’s internal components. Chip materials such as silicon-doped hafnium oxide and metals are likely the SHG and 2PEF sources. SBRs of 8.1 and 10.4 for SHG and 2PEF, respectively, were measured from the image. 7.Discussion and Future WorkWe present an endoscope that enables three optical imaging modalities through the same optical system. The endoscope requirements (as taken from Tables 1–3) and their results are summarized in Table 5. All functional, DAQ, and safety requirements were met, as were most optical requirements. Measurements of lateral resolution from reflectance (low NA) and MPM (high NA) images were poorer than those predicted by optical modeling software. Nonetheless, the lateral resolution of the reflectance modality was still within the required range. Axial resolution for OCM could likely be improved by better calibration, and lateral resolution of MPM by better optical alignment and tighter tolerances on optics manufacturing. Table 5Summary of the requirements along with the results and if the performance passes or fails the criteria.
Future plans include increasing system performance. For example, the piezo harmonic frequencies limit the number of rows that an image can have. By utilizing the first and fourth resonance, the number of pixels in the slow scanning dimension of reflectance and MPM images could be increased. A drawback of using the fourth resonance is that at higher harmonics, the deflection amplitude decreases, requiring higher voltages to increase the amplitude, and possibly impairing the ability to scan the entire FOV. A simpler change would be to acquire signals on both sides of the fast sinusoidal scan, doubling the number of pixels in the slow scanning dimension. Image adjustment to reverse-order every other line would need to be implemented, as well as better calibration to accommodate the minor stable whirling of the fiber, which leads to the fiber tracing an ellipse rather than a line. Simultaneous acquisition of OCM and MPM images is possible but not currently implemented in software. Significant losses occur for high NA modalities at the optical system dichroic filter obscuration. Together with other system losses including imperfections in the coatings and optical alignment, the power on the sample is only 0.9 mW for OCM (89% power loss). The reflected OCM light follows the same high NA path back to the core of the scanning fiber core, with additional losses from the filter obscuration. Although the OCM SBR was reasonable, a higher sensitivity could be achieved with a higher power source. For MPM, the power on the sample was (83% loss), which is low given the system’s fast scanning speed. Imperfections in the optics and alignment lead to a larger spot size. Both low power and larger spot size adversely affected the strength of the generated multiphoton signal. Additionally, the visible wavelength remitted multiphoton signal travels through the optic’s low NA path and is poorly collected in the scanning fiber inner cladding. Increasing the optics quality and alignment leading to a smaller spot size, and/or replacing the custom 100 mW laser with a commercially available 1 W laser at the same pulse duration and repetition rate would deliver a squared increase in generated multiphoton signal.31 Redesigning the coatings to reflect wavelengths would lead to an increase in efficiency of collection of two photon process light and higher SBR in these images. The current system could also provide additional functionality. By replacing the reflectance modality photodiode with a filter and PMT, the same low NA optical path combined with raster scanning and the 12 multimode optical collection fibers could be used for single-photon fluorescence imaging. Since the triplet dichroic filters have a cut-on at , OCT could be performed by changing out the 1300 nm superluminescent diode for one in the 800 nm range. The WDMs would also need to be swapped for ones that worked at this wavelength. The OCT system would then work through the low NA optical path, providing a larger FOV and depth of focus but worse lateral resolution. Beyond system efficiency and source power improvements, next steps include fully packaging and sterilizing the endoscope, and demonstrating its use for imaging of the ovaries and fallopian tubes in an ewe model and in human volunteers already undergoing a laparoscopic procedure. This future work will validate the endoscope by evaluating the instrument’s ability to locate regions with increased vascularity using the reflectance modality, thickening of epithelial layers with OCM, and abnormal cell nucleus and shape with MPM, which are hallmarks of cancer. This endoscope could also be used for other purposes. Most organs that can be reached with a rigid endoscope represent a potential site of use. For example, modification of the endoscope with a shorter hypotube could enable oral cancer screening. AcknowledgmentsThe authors want to acknowledge the support provided by several teams. The Ultrafast Fiber Lasers and Nonlinear Optics group, led by Prof. Khanh Kieu, built the 1400 nm custom-made pulsed laser. INTAQ supplied the expertise needed to program the FPGA allowing the program to achieve software performance. Castor Optics tailored a custom solution to build the WDM/DCFC. Kelli Kiekens, Zuzana Adams, Steven Santaniello, and Harrison Thurgood completed work that served as a precursor or support for this project. Research reported in this publication was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under Award No. 1R01EB020605. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE-1746060. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. The authors declare no conflicts of interest. ReferencesK. A. Cronin et al.,
“Annual report to the nation on the status of cancer, part I: National cancer statistics: annual report national cancer statistics,”
Cancer, 124
(13), 2785
–2800
(2018). https://doi.org/10.1002/cncr.31551 CANCAR 0008-543X Google Scholar
T. W. Sawyer et al.,
“Fluorescence and multiphoton imaging for tissue characterization of a model of postmenopausal ovarian cancer,”
Lasers Surg. Med., 52
(10), 993
–1009
(2020). https://doi.org/10.1002/lsm.23251 LSMEDI 0196-8092 Google Scholar
A. R. Tumlinson et al.,
“In vivo ultrahigh-resolution optical coherence tomography of mouse colon with an achromatized endoscope,”
J. Biomed. Opt., 11
(6), 064003
(2006). https://doi.org/10.1117/1.2399454 JBOPFO 1083-3668 Google Scholar
Y. Li et al.,
“Multimodality endoscopic optical coherence tomography and fluorescence imaging technology for visualization of layered architecture and subsurface microvasculature,”
Opt. Lett., 43
(9), 2074
(2018). https://doi.org/10.1364/OL.43.002074 OPLEDP 0146-9592 Google Scholar
D. Lorenser et al.,
“Dual-modality needle probe for combined fluorescence imaging and three-dimensional optical coherence tomography,”
Opt. Lett., 38
(3), 266
–268
(2013). https://doi.org/10.1364/OL.38.000266 OPLEDP 0146-9592 Google Scholar
H. Yoo et al.,
“Intra-arterial catheter for simultaneous microstructural and molecular imaging in vivo,”
Nat. Med., 17
(12), 1680
–1684
(2011). https://doi.org/10.1038/nm.2555 1078-8956 Google Scholar
O. Nadiarnykh et al.,
“Alterations of the extracellular matrix in ovarian cancer studied by second harmonic generation imaging microscopy,”
BMC Cancer, 10
(1), 94
(2010). https://doi.org/10.1186/1471-2407-10-94 BCMACL 1471-2407 Google Scholar
J. M. Watson et al.,
“Two-photon excited fluorescence imaging of endogenous contrast in a mouse model of ovarian cancer: two-photon excited fluorescence imaging,”
Lasers Surg. Med., 45
(3), 155
–166
(2013). https://doi.org/10.1002/lsm.22115 LSMEDI 0196-8092 Google Scholar
J. M. Watson et al.,
“In vivo time-serial multi-modality optical imaging in a mouse model of ovarian tumorigenesis,”
Cancer Biol. Ther., 15
(1), 42
–60
(2014). https://doi.org/10.4161/cbt.26605 Google Scholar
L. P. Hariri et al.,
“Simultaneous optical coherence tomography and laser induced fluorescence imaging in rat model of ovarian carcinogenesis,”
Cancer Biol. Ther., 10
(5), 438
–447
(2010). https://doi.org/10.4161/cbt.10.5.12531 Google Scholar
M. Kirillin,
“Criteria for pathology recognition in optical coherence tomography of fallopian tubes,”
J. Biomed. Opt., 17
(8), 081413
(2012). https://doi.org/10.1117/1.JBO.17.8.081413 JBOPFO 1083-3668 Google Scholar
T. Wang, M. Brewer and Q. Zhu,
“An overview of optical coherence tomography for ovarian tissue imaging and characterization: an overview of optical coherence tomography,”
Wiley Interdiscipl. Rev. Nanomed. Nanobiotechnol., 7
(1), 1
–16
(2015). https://doi.org/10.1002/wnan.1306 Google Scholar
T. W. Sawyer et al.,
“Quantification of multiphoton and fluorescence images of reproductive tissues from a mouse ovarian cancer model shows promise for early disease detection,”
J. Biomed. Opt., 24 096010
(2019). https://doi.org/10.1117/1.JBO.24.9.096010 JBOPFO 1083-3668 Google Scholar
T. W. Sawyer et al.,
“Three-dimensional texture analysis of optical coherence tomography images of ovarian tissue,”
Phys. Med. Biol., 63 235020
(2018). https://doi.org/10.1088/1361-6560/aaefd2 PHMBA7 0031-9155 Google Scholar
I. T. A. Peters et al.,
“Noninvasive detection of metastases and follicle density in ovarian tissue using full-field Optical coherence tomography,”
Clin. Cancer Res., 22
(22), 5506
–5513
(2016). https://doi.org/10.1158/1078-0432.CCR-16-0288 Google Scholar
W. R. Zipfel, R. M. Williams and W. W. Webb,
“Nonlinear magic: multiphoton microscopy in the biosciences,”
Nat. Biotechnol., 21
(11), 1369
–1377
(2003). https://doi.org/10.1038/nbt899 NABIF9 1087-0156 Google Scholar
R. M. Williams et al.,
“Strategies for high resolution imaging of epithelial ovarian cancer by laparoscopic nonlinear microscopy,”
Translational Oncol., 3
(3), 181
–194
(2010). https://doi.org/10.1593/tlo.09310 Google Scholar
A. A. Tanbakuchi et al.,
“In vivo imaging of ovarian tissue using a novel confocal microlaparoscope,”
Am. J. Obstetr. Gynecol., 202
(1), 90.e1
–90.e9
(2010). https://doi.org/10.1016/j.ajog.2009.07.027 AJOGAH 0002-9378 Google Scholar
D. Hanahan and R. A. Weinberg,
“Hallmarks of cancer: the next generation,”
Cell, 144
(5), 646
–674
(2011). https://doi.org/10.1016/j.cell.2011.02.013 CELLB5 0092-8674 Google Scholar
S. A. Alawi et al.,
“Optical coherence tomography for presurgical margin assessment of non-melanoma skin cancer: a practical approach,”
Exp. Dermatol., 22
(8), 547
–551
(2013). https://doi.org/10.1111/exd.12196 EXDEEY 0906-6705 Google Scholar
J. Yan et al.,
“Real-time optical diagnosis for surgical margin in low rectal cancer using multiphoton microscopy,”
Surg. Endosc., 28
(1), 36
–41
(2014). https://doi.org/10.1007/s00464-013-3153-7 Google Scholar
K. Visvanathan et al.,
“Diagnosis of Serous tubal intraepithelial carcinoma based on morphologic and immunohistochemical features: a reproducibility study,”
Am. J. Surg. Pathol., 35
(12), 1766
–1775
(2011). https://doi.org/10.1097/PAS.0b013e31822f58bc Google Scholar
E. Hemmer et al.,
“Exploiting the biological windows: current perspectives on fluorescent bioprobes emitting above 1000 nm,”
Nanosc. Horiz., 1
(3), 168
–184
(2016). https://doi.org/10.1039/C5NH00073D Google Scholar
K. Kiekens et al.,
“Effect of an added mass on the vibration characteristics for raster scanning of a cantilevered optical fiber,”
J. Eng. Sci. Med. Diagn. Ther., 4 021007
(2021). https://doi.org/10.1115/1.4050691 Google Scholar
D. Vega et al.,
“Use of embedded and patterned dichroic surfaces with reflective optical power to enable multiple optical paths in a micro-objective,”
Appl. Opt., 59
(22), G71
(2020). https://doi.org/10.1364/AO.391654 APOPAI 0003-6935 Google Scholar
C. M. Lee et al.,
“Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,”
J. Biophotonics, 3
(5-6), 385
–407
(2010). https://doi.org/10.1002/jbio.200900087 Google Scholar
T. Kinoshita et al.,
“A novel laser fiberscope for simultaneous imaging and phototherapy of peripheral lung cancer,”
Chest, 156
(3), 571
–578
(2019). https://doi.org/10.1016/j.chest.2019.04.010 CHETBF 0012-3692 Google Scholar
D. Vega et al.,
“A coregistered multimodal imaging system for reflectance, multiphoton, and optical coherence microscopy,”
Proc. SPIE, 11634 116340Q
(2021). https://doi.org/10.1117/12.2576954 PSISDG 0277-786X Google Scholar
X. Attendu et al.,
“Simple and robust calibration procedure for k-linearization and dispersion compensation in optical coherence tomography,”
J. Biomed. Opt., 24
(5), 056001
(2019). https://doi.org/10.1117/1.JBO.24.5.056001 JBOPFO 1083-3668 Google Scholar
S. Mehravar, B. Cromey and K. Kieu,
“Characterization of multiphoton microscopes by the nonlinear knife-edge technique,”
Appl. Opt., 59 G219
–G224
(2020). https://doi.org/10.1364/AO.391881 APOPAI 0003-6935 Google Scholar
C. Boudoux, Fundamentals of Biomedical Optics: From Light Interactions with Cells to Complex Imaging Systems, Pollux, Montréal
(2018). Google Scholar
BiographyDavid Vega obtained his PhD from James C. Wyant College of Optical Sciences at the University of Arizona. He was awarded the NSF GRFP fellowship due to his original research and has received multiple recognitions for his work in areas related to the optical engineering field, such as astronomy, physics, optical design, and optical manufacturing. |