1.

Introduction

Photoacoustic (PA) imaging is an emerging technique capable of visualizing the vasculature deep within the tissue by exploiting intrinsic molecular contrast.^1,2 Miniaturized PA probes have many potential applications in intraoperative imaging and minimally invasive surgical guidance. These include the in situ assessment of cancerous lesions in the abdominal organs, identification of tumor margins during laparoscopic liver surgery, guidance of needle biopsies, and treatment monitoring of photocoagulation therapy. These applications often require a forward-viewing configuration to visualize the tissue directly ahead of the probe tip. However, the majority of previously reported PA endoscopic probes tend to be side viewing.^3–7 Of the few forward-viewing probes that have been demonstrated, almost all employ optical resolution microscopy (OR-PAM)^8–11 in which the imaging depth is restricted to $<1\mathrm{mm}$. For greater penetration depths, widefield photoacoustic tomography (PAT) mode is desirable.¹ Unlike OR-PAM, which employs a scanning focused excitation laser beam and a single static detector, PAT requires widefield illumination and mapping of the PA wavefield at multiple spatial points. This poses several technical challenges. First, for high resolution three-dimensional (3D) PAT, a high-density two-dimensional (2D) ultrasound array that provides fine spatial sampling ($<100\mu \mathrm{m}$) and wide detection bandwidth (tens of MHz) is required but is challenging to achieve with conventional piezoelectric detection technology.¹² Capacitive micromachined ultrasonic transducers (CMUT) fabrication technology can produce high-density arrays^13–16 but lacks the large detection bandwidth for high-resolution PAT imaging. Second, efficient delivery of the excitation light to the tissue requires delivering the widefield illumination through the detector to the tissue. This necessitates a transparent detector array, but most conventional piezoelectric receivers and CMUTs are opaque.

The Fabry–Pérot (FP) ultrasound sensing concept¹⁷ can address these limitations by providing the requisite wide-bandwidth, fine spatial sampling and an optically transparent sensor head. Miniature, forward-viewing 3D PAT endoscopic probes based on this type of sensor have been demonstrated.^18,19 These devices employ a high density coherent fiber bundle to read out the FP sensor, which offers a high degree of miniaturization¹⁹ and the flexibility¹⁸ for minimally invasive applications that require tortuous access. In this paper, we describe an endoscopic probe based on a different design. Instead of using a fiber bundle to read out the sensor, a free-space scanning beam delivered via a miniature MEMs scanner and an optical relay is used. This approach avoids the need (i) for a scanner system that can provide sub-micron optical positional accuracy to couple the interrogation laser beam into the cores of the fiber bundle and (ii) to overcome the limitations of commercially available fiber bundles, which tend not to be designed for single-mode operation at the 1500 to 1600 nm FP sensor interrogation wavelength range and can result in additional noise.²⁰ Although these challenges are not fundamental limitations, they do make free-space interrogation a more attractive proposition for applications in which a high degree of miniaturization and flexibility of the fiber-bundle based probes are not required. For example, for an application such as guiding laparoscopic surgery of the liver, a rigid probe with an outer diameter in the 10 to 15 mm range (which is similar to that of a rigid laparoscopic ultrasound probe) would be acceptable and amenable to a free-space FP sensor read-out scheme.

A forward-viewing rigid endoscope based on the above free-space read-out design that provides high-resolution 3D PAT imaging is described. An additional novel aspect is that the probe is designed to provide an integrated video rate white light imaging capability that enables the target tissue ahead of the probe tip to be visualized in real time. The dual-modality approach enables a comprehensive in situ assessment of tissue as it enables both visual inspection of the tissue surface and 3D imaging of the subsurface vascular anatomy.

2.

Experimental Setup

A schematic of the dual-modality endoscope is shown in Fig. 1. The probe comprises a stainless-steel tube of length 270 mm and external diameter 10 mm with an optically transparent FP ultrasound sensor located at its distal end. To acquire PAT images, pulsed excitation laser light at 750 nm is delivered along the stainless-steel tube and through the FP sensor to illuminate the tissue. The resulting generated PA waves are recorded in 2D by optically scanning the FP sensor with a focused 1550 nm laser beam. A 3D PAT image is then reconstructed from the measured data. To acquire white light images, an array of optical fibers embedded within the wall of the stainless-steel tube is used to deliver widefield visible light to the tissue. The back-scattered light is then transmitted back through the FP sensor, relayed along the length of the tube, and imaged onto a miniature CMOS camera (C) at the proximal end.

Fig. 1

Schematic of the dual-modality PA endoscope. EL, excitation laser; SIL, sensor interrogation laser; WL, constant intensity white light source; SM, 2D MEMS scanning mirror; C, CMOS camera. (a) Cross-section of the endoscope tube, which shows white light illumination fibers. (b) Miniature 2D MEMS mirror used to optically scan the FP sensor with an interrogation laser beam. (c) Schematic showing the cross-section of the FP sensor, which comprises a polymer spacer sandwiched between two dichroic mirror coatings.

The FP ultrasound sensor comprises a thin film multilayer structure consisting of a $25\mu \mathrm{m}$ thick Parylene C spacer sandwiched between a pair of dichroic mirrors that acts as an FP etalon. The mirrors are designed for high transmission in the visible and near infrared spectral region (580 to 1250 nm) and high reflectivity in the short-wave spectral region (1500 to 1600 nm). This enables both the 750 nm PA excitation laser light and the white light used to acquire the video image to be delivered through the sensor and on to the tissue sample. At the same time, the mirrors provide the necessarily high reflectivities over the FP sensor interrogation laser wavelength range (1500 to 1600 nm) for the thin film structure to act as an etalon. By tuning the interrogation laser wavelength to the edge of the etalon transfer function,¹⁷ acoustically induced optical thickness changes in the spacer thickness produce a corresponding reflected power modulation of the interrogation beam, which enables the detection of the PA waves. To record the PA wavefield at different points across the sensor, a miniature two-axis electrostatic MEMs based scanner (Mirrorcle Technologies Inc.) located at the proximal end of the probe is used to scan a focused interrogation beam of a $62.5\mu \mathrm{m}$ diameter across its surface via a $4f$ optical relay system and a scanning lens. The optical relay serves to image the scanning mirror pivot point onto the back focal plane of the scanning lens to enable the FP sensor to be located 27 cm from the MEMs scanner. The interrogation beam is reflected from the FP sensor, back through the relay system and MEMs scanner, and on to a fiber coupled InGaAs photodiode-transimpedance amplifier unit with a 70 MHz bandwidth. The output of the photodiode is connected to a digitizer with an analog bandwidth of 60 MHz to record the detected PA signals. A tunable (660 to 2200 nm) fiber-coupled optical parametric oscillator-based laser system (Innolas GmbH), which provides 7 ns pulses at a 200 Hz pulse repetition frequency (PRF), is used as the PA excitation source. A short-pass dichroic mirror combines the excitation and sensor interrogation beams at the proximal end of the endoscope. The excitation beam then passes through the optical relay and exits the endoscope tip as a near-collimated 7 mm diameter beam. To reconstruct a 3D PAT image from the detected PA signals, a time reversal algorithm²¹ is used.

A constant intensity LED source (Karl Storz Power LED 175) is used to provide the illumination for the white light endoscopy images. The LED output is coupled into a bundle of 128 multi-mode optical fibers ($225/200\mu \mathrm{m}$ core/cladding diameters; Thorlabs Inc.) that were embedded within the walls of the stainless-steel tube. These fibers illuminate the target tissue at the tip of the endoscope, and the back-scattered light is relayed back through the optical relay system and imaged onto a 40 Hz CMOS camera (Basler acA-1920-40c). A 25 mm fixed focal length camera lens in combination with a $-200\mathrm{mm}$ focal length plano-concave lens is used to set the widefield imaging focus at a distance of 5 mm from the probe tip. The system was designed to provide an overall magnification of 2.5×, a 7 mm diameter field-of-view, and a 5 mm depth of focus.

3.

Results

The PAT imaging performance of the dual-modality endoscope was evaluated in terms of noise-equivalent-pressure, detection bandwidth, field-of-view, and lateral and axial resolution. The rms NEP was measured at 8100 spatial positions on the FP sensor, and the modal value was 150 Pa over 20 MHz detection bandwidth. The $-3\mathrm{dB}$ detection bandwidth of the FP sensor, which depends on the optical thickness of the polymer spacer, was estimated to be 35 MHz using an experimentally validated numerical frequency response model.²² The field-of-view and spatial resolution were evaluated by imaging a multi-layer black polymer ribbon phantom immersed in a scattering 1% intralipid solution. The ribbons were $20\mu \mathrm{m}$ thick and sufficiently wide to approximate as an absorbing step in the lateral direction.

A 3D PAT image of the phantom was acquired using a 750 nm excitation wavelength, $5\mathrm{mJ}/{\mathrm{cm}}^2$ fluence, and scanning a 7 mm diameter area in steps of $75\mu \mathrm{m}$ with a total acquisition time of 40 s. Figure 2(a) shows a cross-section along the $x\text{-}z$ plane from the reconstructed 3D PA image of the ribbon phantom. The edge response function was evaluated for each ribbon cross-section and interpolated over the $x\text{-}y$ FOV to estimate the lateral spatial resolution [Fig. 2(b)]. The lateral spatial resolution was $70\mu \mathrm{m}$ at a depth of 1.5 mm from the tip of the probe and decreased to $100\mu \mathrm{m}$ at a depth of 7 mm. The vertical resolution was estimated at $28\mu \mathrm{m}$ based on the bandwidth of the sensor, and it is approximately spatially invariant over the FOV.

Fig. 2

Phantom images: (a) $x\text{-}z$ cross-section extracted from the reconstructed 3D PAT image of a multi-layer ribbon phantom that shows a cross-section over the probe field of view. (b) A contour plot showing the variation in the lateral spatial resolution, obtained by evaluating the edge spread function from each ribbon feature. Dual-modality images of arbitrarily shaped phantoms. (c), (d) White light images of a synthetic hair knot and leaf skeleton phantom. (e)–(h) Horizontal and vertical projections from the 3D PAT images of the phantoms.

The dual-mode imaging capability of the endoscope was first evaluated by imaging arbitrarily shaped absorbing phantoms. Figures 2(c) and 2(d) show the white light images of a synthetic hair knot and leaf skeleton phantom, which were immersed in deionized water. PAT images of these phantoms were acquired using a 750 nm excitation wavelength and $5\mathrm{mJ}/{\mathrm{cm}}^2$ fluence. The reconstructed PAT images of the phantoms are shown in Figs. 2(e)–2(h), as maximum intensity projections along the horizontal ($x\text{-}y$) and vertical plane ($x\text{-}z$). The structural features of the synthetic hair knot and the intricate veins of the leaf skeleton phantom are accurately reproduced in the PAT images, showing good correspondence with their respective white light endoscopic images.

The dual-modality imaging capability in biological media was demonstrated by examining the abdominal organs of an ex vivo mouse, made accessible via a large incision in the abdomen. This is best illustrated in Video 1 (available online), which shows how the white light imaging capability is used to navigate the probe to the organ of interest. Once the probe is correctly positioned, it is gently pressed against the tissue, and a 3D PAT image is acquired. Figure 3 shows selected images from Video 1. The top panel shows the white light images of the spleen, liver, and kidney. The 3D PAT images of the organs from the same locations are shown in the bottom panel of Fig. 3. The boundaries of the spleen are clearly visualized in the horizontal and vertical projections shown in Figs. 3(d) and 3(g). Because the spleen has a fine network of sinusoids and stores a significant amount of blood, it produces a strong and homogeneous PA contrast. The PAT images of the liver show the outline of a lobe and its underlying vascular anatomy in both projections, as shown in Figs. 3(e) and 3(h). The PAT images of the kidney clearly visualize the renal vasculature. The vertical projection in Fig. 3(i) shows the complete longitudinal view of the kidney and its renal vasculature up to 5 mm in depth.

Fig. 3

Dual-modality images of ex vivo mouse abdominal organs in situ. (a)–(c) The top panel shows white light endoscopic images of the spleen, liver, and kidney. (d)–(i) Horizontal and vertical projections from the 3D PAT images showing the underlying vascular anatomy of respective organs. Excitation wavelength: 760 nm. (Video 1, MP4, 704 KB [URL: https://doi.org/10.1117/1.JBO.29.2.020502.s1].)

4.

Conclusion

In summary, a novel dual-modality endoscope that combines PAT imaging and conventional white light videoscopy in a forward-viewing configuration has been presented. High-resolution PAT imaging of biological tissues was demonstrated by visualizing the 3D vascular structures of abdominal organs from an ex vivo mouse including spleen, liver, and kidney. In addition, white light images of the tissue surface were acquired, illustrating the dual modal capability of the probe.

The novel aspects of the probe are the use of an MEMs based scanner and optical relay system to read out the FP sensor. The small size and low mass of the MEMs scanner allows the proximal end of the probe to be sufficiently compact and lightweight for practical endoscopic use. This would not be possible using the much bulkier and heavier galvanometer-based scanners used in previous free-space FP sensor read-out schemes.¹⁷ The relay is also a key enabling design feature. By mapping the optical “pivot point” from the MEMs scanner to the distal end of the probe, it allows the FP sensor to be located at the end of a tube that is sufficiently long (27 cm) for endoscopic use. By contrast, in previous free-space FP scanners, the scanner-sensor distance was much less, typically 5 cm, and thus insufficient for an endoscopic implementation.

The design of the probe offers several inherent advantages. These include the broadband ultrasound frequency response of the FP sensor and fine spatial sampling, which provides high-resolution PAT images and a forward-viewing configuration for surgical guidance applications. The integrated white light imaging capability is useful, not only for providing tissue characterization information but also for navigating to the anatomical area of interest prior to acquiring a PAT image. Moreover, the concept is not limited to integrating white light imaging alone. It would also be possible to integrate other optical imaging techniques, such as narrowband, hyperspectral, fluorescence imaging, or optical coherence tomography.²³ There is further potential to incorporate a pulse-echo ultrasound imaging capability by depositing a dichroic absorbing coating on to the FP sensor that acts as a laser-generated ultrasound transmitter.²⁴

Although feasibility has been demonstrated, translating the concept into practical in vivo clinical use will require further development, most obviously to reduce the current PAT acquisition time of 40 s. This could be addressed using a multi-beam scanner²⁵ to parallelize the sensor read-out and the use of higher PRF excitation lasers and compressed sensing techniques.²⁶ In combination, these approaches offer the prospect of achieving real-time 3D image frame rates.

It is anticipated that this type of probe could ultimately find several clinical endoscopic applications in which a high level of miniaturization is not a primary requirement and a rigid device can be accommodated, for example, in keyhole laparoscopic surgery to remove tumors in the liver and other abdominal organs. Surgical instruments, endoscopes, and rigid laparoscopic ultrasound probes are commonly delivered via ports introduced in the abdominal wall. The diameter of these ports is typically in the 5 to 10 mm range and thus could accommodate our probe, which could be used to visualize the margins of tumors based on their vascular contrast to help guide their removal by surgery or ablative methods.