2.1.

Three-Dimensional Fluorescence Imaging System

The multimodal system comprises two DHMs for 3-D fluorescence and phase imaging. Figure 1(a) shows the schematic of the common-path off-axis fluorescent digital holographic system for 3-D FI recently developed by our group.¹⁰ This fluorescent microscope is accomplished by embedding a focusing lens with a diffraction grating, as shown in Fig. 1(b), onto a phase-mode spatial light modulator (SLM). This pattern allows splitting the incident fluorescent light from the object into two light waves with slightly different propagation directions in order to achieve off-axis interference. Figure 1(c) shows the focused image of the fluorescent beads of size $\sim 10.4\mu \mathrm{m}$ when no pattern is displayed on the SLM, and Fig. 1(d) depicts the two fluorescent lights: one unmodulated and another modulated by the pattern [shown in Fig. 1(b)] displayed on the SLM. These two wavefronts interfere at the image sensor plane and form a fluorescent digital hologram, as shown in Fig. 1(e), by the help of a linear polarizer, when the sample (fluorescent beads) is moved in the ($\pm$) $z$ direction by a small distance, say $-80\mu \mathrm{m}$, as in this case.

Fig. 1

Incoherent digital holographic system for 3-D FI. (a) Schematic representation of the common-path off-axis digital holographic setup. (b) The phase pattern displayed on the SLM, (c) focused image of the fluorescent beads, and an enlarged view of one bead is also shown sideways, (d) image obtained when the pattern shown in Fig. 1(b) displayed on the SLM, and an enlarged view of one bead is also shown sideways, (e) recorded fluorescent hologram obtained by moving the sample in the $z$ direction by $-80\mu \mathrm{m}$, (f) reconstructed image of the fluorescent beads, and (g) normalized intensity plot for the recorded and reconstructed selected fluorescence beads across the line AB, marked in (f).

If the radii of two wavefronts are denoted as $r_m$ and $r_u$ by setting the focal length of $f_{\text{SLM}}=f_0$ and infinity, respectively, the reconstructed distance from the image sensor plane is described as $z_h=r_m{r}_u/r_m-r_u$. From the recorded fluorescence hologram, the 3-D object information can be reconstructed by using the Fresnel propagation algorithm. Figure 1(f) shows the reconstructed image of the fluorescent beads, and the normalized intensity plot is shown in Fig. 1(g) for the recorded and reconstructed selected fluorescence beads across the line AB, marked in Fig. 1(f). The signal-to-noise ratio (SNR) for the recorded image ($\mathrm{SNR}=30.4445$) and the reconstructed image ($\mathrm{SNR}=29.8611$) are almost the same. This shows the good performance of the system.

2.2.

Three-Dimensional Phase Imaging System

The phase imaging is accomplished by a single-shot common-path off-axis DHM using a cube beam splitter. Figure 2 shows the schematic of the experimental setup of the proposed beam splitter-based common-path DHM. The collimated laser light (He-Ne laser, $\lambda =632.8\mathrm{nm}$) transilluminates the sample mounted on a motorized translational stage. The light transmitted through the sample is collected by a microscopic objective lens [$40\times$ magnification, numerical aperture (NA) = 0.65] and collimated by the lens $L2$ ($\text{focal length}=100\mathrm{mm}$). The collimated light is focused by $L3$ ($\text{focal length}=150\mathrm{mm}$) and allowed to pass through a cube beam splitter, BS, which divides the incident object beam into two beams. Since both the beams carry object information, their interference on the image sensor may bear the overlapping issue. To avoid the overlapping, one beam is spatially filtered at the Fourier plane by using a pinhole ($50\mu \mathrm{m}$, diameter), mounted on a 3-D translational stage. The pinhole acts as a spatial filter and is positioned in such a way that it blocks the higher frequency components and allows only the DC component of the beam to pass and converted into a spherical reference beam, in which all the object information is erased. After passing through a lens $L4$, the reference wave becomes a tilted plane wave. This is good for ideal off-axis holography. The orientation of the beam splitter is made in such a way that the two divided beams have sufficient lateral separation. The interference of the object beam and the reference beam is recorded by a CMOS camera (Sony Pregius IMX 249, sensor format: $1920\times 1200\mathrm{pixels}$, pixel size of $5.86\mu \mathrm{m}$).

Fig. 2

Schematic representation of the proposed common-path off-axis DHM setup. BS, Beam splitter; SF, spatial filter; $L1$ to $L4$, lenses.

3.1.

Common-Path Digital Holographic Microscopy

First, the performance of the single-shot common-path off-axis DHM is demonstrated. The proposed system, owing to its common-path configuration, shows high temporal stability compared to the conventional two-beam configuration of DHMs. To measure the temporal stability of the system, a series of holograms, without the presence of the object, are recorded at the rate of 40 frames per second, for 100 s without any vibration isolation. Then, the phase distributions are reconstructed numerically for all the 4000 holograms, and the phase difference distributions are calculated for all the frames by comparing the reconstructed phase distributions to that of the first recorded hologram. The standard deviation of the phase difference for 10,000 random pixel points in the same area of every phase difference distribution is calculated. Figure 3 shows the histogram of the standard deviation of the proposed setup indicating that a mean fluctuation is 0.0098 rad. On the other hand, the mean variation of phase is $\sim 0.2\mathrm{rad}$ for the traditional two-beam (e.g., Mach–Zehnder) interferometer.³ Therefore, the proposed setup shows improved temporal stability in comparison to its counterparts.

Fig. 3

Temporal stability of the proposed setup. Histogram of the standard deviation between reconstructed phase distributions.

Then the imaging capability of the proposed DHM is demonstrated by performing several experiments on objects, such as a United States Air Force (USAF) resolution chart, microsphere beads, and 3-D biological sample. Only a single hologram, in the presence of the object, is recorded and is further used to extract the object intensity and phase distribution. Figures 4Fig. 5–6 show the experimental results on the USAF resolution chart. Figures 4(a) and 4(b) show the recorded hologram and the enlarged view of the selected region of the recorded hologram, respectively. Figures 4(c)–4(e) show the Fourier spectrum of the recorded hologram, filtered $+1$ order, and $+1$ order centered, respectively. A spherical phase curvature is introduced in the object wave after passing through the microscopic objective and that must be compensated. The numerical phase aberration compensation method based on the PCA is utilized in order to accurately recover the phase information.⁴⁸ This method is based on the decomposition of the phase map into a set of values of uncorrelated variables. These variables are called principal components and from the first principal component the aberration term is retrieved. The first principal component of the exponential term of the filtered hologram is estimated by using a singular value decomposition. Then the linear and the quadratic coefficients can be identified using least-squares fitting and their conjugate is multiplied with the filtered hologram in order to obtain aberration-free phase distribution. These steps involved extracting aberration-free phase distribution using the PCA method are demonstrated in Figs. 5(a)–5(c). Figure 5(a) shows the raw sampled phase distribution carrying the phase aberration. Figure 5(b) shows the obtained phase aberration distribution using the PCA method. The conjugate of this aberration term is multiplied with the sampled hologram [Fig. 5(a)] and results in the modification of the original region of the spectrum. The new modified spectrum is shown in Fig. 5(c). Then the numerical reconstruction process is performed to obtain the aberration-free phase distribution. Figures 6(a) and 6(b) show the retrieved intensity and phase distribution corresponding to the recorded hologram shown in Fig. 4(a). The PCA-based phase aberration compensation method does not require prior knowledge either of the object or of the setup and shows efficient performance with promising results.⁴⁸

Fig. 4

Common-path off-axis DHM results of USAF resolution chart: (a) recorded digital hologram, (b) the enlarged view of the selected region showing the fringes, (c) Fourier spectrum of the hologram, (d) filtered $+1$ order, and (e) $+1$ order centered (contrast adjusted).

Fig. 5

(a) Raw sampled phase, (b) conjugate phase extracted, and (c) compensated spectrum.

Fig. 6

(a) The reconstructed intensity image corresponding to the recorded hologram shown in Fig. 4(a) and (b) unwrapped phase map.

In the next experiments, the capability of the proposed DHM system is demonstrated on a 3-D biological sample, the living plant cells. As the observation target, we used the moss Physcomitrella patens (Physcomitrella).⁴⁹ In Physcomitrella, the genome is sequenced,⁵⁰ the cell identity is clear,⁴⁹ and the body size is compact, all of which make Physcomitrella one of the model organisms for modern biology. Therefore, Physcomitrella is suitable as the observation target using the proposed system. Figure 7 shows the experimental results of the proposed DHM on the living plant cells of protonemata, the hypha-like structure of Physcomitrella. The recorded hologram of the 3-D sample is shown in Figs. 7(a) and Figs. 7(b)–7(e) show the wrapped phase maps of the four different in-focus planes obtained from the recorded hologram, where blue arrows indicate the in-focus regions. Video 1 depicts the movie of the recovered in-focus phase imaging from one plane [corresponding to the in-focus plane of Fig. 7(b)] to another [to the in-focus plane of Fig. 7(e)].

Fig. 7

Common-path off-axis DHM experimental results of living plant cells of protonemata, the hypha-like structure of Physcomitrella: (a) digital hologram and (b)–(e) some of the retrieved wrapped phase maps at different in-focus planes (the in-focus region is indicated by the blue arrows). (Video 1, mp4, 6442 KB [URL: https://doi.org/10.1117/1.JBO.25.3.032010.1]).

3.2.

Multimodal System

In this section, we demonstrate the performance of the multimodal imaging system developed by combining the common-path off-axis DHM and the common-path off-axis FDHM, in order to retrieve simultaneously both the 3-D phase and 3-D fluorescence images. Figure 8 shows the schematic of the proposed multimodal system. Here, we use two image sensors for fluorescence and phase imaging. For the applications of biological samples, the optical power difference is too much to record both holograms by a single image sensor because the fluorescence light is too weak to avoid phototoxicity. However, two cameras create an image registration problem. In other applications, such as material sciences, a single image sensor is ideal for fabricating an integrated multimodal system. The performance of the proposed multimodal system is demonstrated by performing experiments on microsphere fluorescent beads and fluorescent protein-labeled living plant cells. A blue laser (wavelength, $\lambda =473\mathrm{nm}$) is used as a light source for the incoherent DHM system, to excite the fluorescent object used in the study. The laser beam is spatially filtered, collimated, and then reflected from a dichroic mirror (DM1, Thorlabs, DMLP490R) and entered into the objective lens ($40\times$ magnification, $\mathrm{NA}=0.65$). The incident laser light excites the fluorescent object. First, an experiment is performed on microsphere fluorescent beads of size $\sim 10.4\mu \mathrm{m}$. The beads, mounted on the 3-D translational stage, are excited by the collimated laser beam in an epi-illumination configuration. The fluorescent beads emit the yellow fluorescence with wavelengths ranging from 550 to 600 nm. This fluorescent light travels back through the objective lens, transmits through DM1, and is reflected from another dichroic mirror, DM2 (Thorlabs, DMLP605R). The Fourier transform of the object beam located at the focused plane of the objective lens is projected onto the plane of the phase-mode SLM (Holoeye pluto, $1920\times 1080\text{pixels}$, $8\mu \mathrm{m}$ pixel pitch, phase-only modulation) by a $4f$ relay system ($L4\to L5$). Here, the back focal plane of the objective lens is imaged on the phase-mode SLM. A lens function with focal length $f_{\text{SLM}}=800\mathrm{mm}$ and a diffraction grating function with grating period $d_h=300\mu \mathrm{m}$ were displayed onto the SLM, which generates two distinct wavefronts at respective angles of the incident beam. These two wavefronts are then imaged by a tube lens ($\text{focal length}=200\mathrm{mm}$) and allowed to interfere on to the faceplate of the electron multiplying charge-coupled device (EMCCD) sensor (Andor iXon 888, sensor format: $1024\times 1024\text{pixels}$, pixel size of $13\mu \mathrm{m}$, sensor diagonal of 18.8 mm) and hence form a fluorescent digital hologram. A linear polarizer is placed before the EMCCD sensor in order to allow the interference of two beams. The volume size of the 3-D fluorescence system we demonstrated in the experiments is $330\times 330\times 100{\mu \mathrm{m}}^3$. The axial size can be more extended. The axial measurable size will be determined by setting the required resolution to be reconstructed.

Fig. 8

Optical schematic of the proposed multimodal system for the measurement of the 3-D fluorescence and the 3-D phase of the specimen.

The proposed common-path off-axis DHM system for phase imaging uses a He-Ne laser ($\lambda =632.8\mathrm{nm}$) to transilluminate the microsphere fluorescent beads mounted on the translational stage. The collimated beam illuminates the microsphere beads and is subsequently magnified by the microscope objective and collimated using the lens $L4$. The object beam passes through both the DMs and again focused by using the lens $L6$. This object beam is divided into two beams by using the beam splitter, BS, in which one beam behaves as an object beam carrying the object information and the other beam is spatially filtered at its Fourier plane by using a pinhole of size $50\mu \mathrm{m}$ and serves the reference beam. The interference of the reference and object beams is recorded by a CMOS camera (Sony Pregius IMX 249, sensor format: $1920\times 1200\text{pixels}$, pixel size of $5.86\mu \mathrm{m}$). The volume size of the fabricated phase imaging system is $280\times 176\times 80{\mu \mathrm{m}}^3$. The axial size can also be more extended. The axial measurable size will be determined by setting the required resolution to be reconstructed.

Figure 9 shows the experimental results of the proposed multimodal system on microsphere fluorescent beads of size $\sim 10\mu \mathrm{m}$. Figure 9(a) shows the phase hologram recorded by common-path off-axis DHM. From this single-phase hologram, the phase information is retrieved by the PCA-based phase aberration compensation method. Figures 9(b) and 9(c) show the retrieved wrapped and 2-D unwrapped phase maps, respectively, of the fluorescence beads. The 3-D phase distribution of a selected bead is shown in Fig. 9(d). Concurrently, the fluorescent beads are imaged by FDHM. Figure 9(e) shows the focused image of the microsphere fluorescent beads. Then, the stage is moved in the $z$ direction by $80\mu \mathrm{m}$, and a fluorescent digital hologram, as shown in Fig. 9(f), is recorded. The fluorescent hologram is recorded by projecting a lens function and a diffraction grating function on to the SLM. A bandpass filter centered at $575\pm 12.5\mathrm{nm}$ was placed in front of an EMCCD sensor in order to improve the fringe visibility of the holograms. Figure 9(g) shows the reconstructed image of the fluorescent beads retrieved from the recorded digital fluorescent hologram by the Fresnel propagation algorithm at the reconstruction distance of 1009 mm.

Fig. 9

Experimental results of the multimodal system of microsphere beads. Phase imaging results: (a) phase hologram, (b) wrapped phase distribution, (c) 2-D unwrapped phase map, and (d) 3-D unwrapped phase distribution of a selected bead. FI results: (e) original focused image of the fluorescent beads, (f) fluorescent digital hologram obtained by moving the beads by $80\mu \mathrm{m}$ along the $z$ direction, and (g) reconstructed image of the fluorescent beads.

In the next experiment, we experimentally demonstrate the 3-D fluorescence and 3-D phase imaging capability of the proposed multimodal system on living plant cells of Physcomitrella, where the “Citrine” yellow fluorescent protein gene (YFP)⁵¹ was inserted into a histone H3.3 locus (Pp3c18_14481⁵⁰) (see Fig. S1 and Supplementary text in the Supplementary Material, for more details), resulting in strong expression of H3.3-YFP in the nuclei.

Figures 10(a)–10(c) show the various focused fluorescence images of the nuclei of the living plant cells distributed in the 3-D space. The yellow arrow, in these figures, indicates the focused nuclei. The focused plane corresponding to Fig. 10(c) is moved by $60\mu \mathrm{m}$ in the $z$ direction and a fluorescent digital hologram is recorded, as shown in Fig. 10(d), by projecting a lens function (with a focal length of 800 mm) and a diffraction grating function (with a grating period of $300\mu \mathrm{m}$) on to the SLM. Figures 10(e)–10(g) show the reconstructed focused images corresponding to Figs. 10(a)–10(c), obtained at 660, 825, and 1030 mm, which correspond to 60, 70, and $80\mu \mathrm{m}$, respectively, in the object space. The phase imaging results of the same plant cells are shown in Figs. 10(h)–10(k). Figure 10(h) shows the recorded phase hologram, and the unwrapped phase maps at three different focal planes are shown in Figs. 10(i)–10(k). Chloroplasts, which strongly delay the phase⁵² and thus appear as dark spots in Figs. 10(i)–10(k), are clearly imaged near the focal plane. Taken together, the proposed multimodal approach can retrieve the 3-D structure of the cells as well as the 3-D position of the fluorescent protein-labeled nuclei in living plant cells. These results show the high potential of the proposed system to observe the 3-D behavior of living cells in a single-shot measurement.

Fig. 10

Experimental results of the multimodal system of living protonema cells of Physcomitrella. FI results: (a)–(c) original focused images of the nuclei ($670\times 1024\text{pixels}$). The yellow arrow, in these figures, indicates the focused nuclei, (d) fluorescent digital hologram, and (e)–(g) reconstructed focused images of the nuclei correspond to Figs. (a)–(c). Phase imaging results: (h) phase hologram and (i)–(k) 2-D unwrapped phase maps corresponding to three focused planes.

Further, the 3-D live fluorescence and phase imaging of the moving fluorescent beads in a volume is also demonstrated by the proposed multimodal system. Figures 11(a) and 11(b) show the retrieved unwrapped phase and reconstructed fluorescence image of the moving bead, respectively, at a time instant ($t=1.21\mathrm{s}$), obtained from the digital phase and fluorescence holograms recorded by the multimodal system. The movie of the retrieved unwrapped phase of the moving microsphere bead is shown in Video 2, whereas the movie of the reconstructed fluorescence images of the same bead is shown in Video 3.

Fig. 11

3-D live phase and FI results: (a) unwrapped phase map and (b) fluorescence image of the moving fluorescent bead at $t=1.21\mathrm{s}$. Also, see Video 2 (mp4, 5984 KB [URL: https://doi.org/10.1117/1.JBO.25.3.032010.2]) (retrieved unwrapped phase distribution) and Video 3 (mp4, 75 KB [URL: https://doi.org/10.1117/1.JBO.25.3.032010.3]) (reconstructed fluorescence image).