2.1.

Structured Illumination Microscope System

A Kohler illumination-based optical system was configured to operate SIM. The warm white LED was projected onto the SLM containing the grid pattern. Light reflected from the SLM was reflected by a beam splitter and transmitted through the band-pass filter to improve the grid pattern contrast.^3,11 The filtered light was reflected into the objective by a beam splitter to form the grid pattern image of the SLM onto the sample. Optical sectioning was performed by the phase shift of a sinusoidal wave in the grid pattern, and the phase shift was electrically implemented in the SIM based on the SLM. The intensity of the samples is modulated by the square-law detection algorithm:

While the samples loaded on the linear stage were axially scanned with the phase shift, the axial intensity was obtained by optical sectioning performed by the square law detection. The focal axial position was determined by the peak axial intensity in each pixel of the camera. The 3-D height map can be obtained by calculating the focal axial position in the Cartesian coordinate system with respect to all the pixels of the camera; this provides the height information of the sample. The performance of the SIM can be improved by increasing the axial response. Hence, as the full width at half maximum (FWHM) of the axial response is narrower, the SIM provides more precise height information.

For SLM-based SIM, the SIM can be optimized by the maximum optical sectioning strength at a normalized spatial frequency ($\tilde{v}$) close to unity in Eq. (2):^2,3

2.2.

Analysis of the Frequency Spectrum

For an incoherent illumination system, the detected image intensity is the result of the convolution of the light intensity ($I_s$) from the sample and the point spread function (PSF) ($h$) of the imaging system:

When the grid pattern is projected uniformly onto the sample, as illustrated in Fig. 1(a):

Fig. 1

(a) One-dimensional (1-D) grid pattern in spatial domain, (b) 1-D grid pattern in the frequency domain, (c) raw image with the frequency spectrum in single optical sectioning, (d) sectioned image with the frequency spectrum in single optical sectioning, and (e) raw image with the frequency spectrum in multiple optical sectioning.

The first term is the detected image of the frequency spectrum cutoff by low-pass filtering and has no high-frequency spectra, and the second and third terms are the detected images of the high-frequency spectra, which represent small structures in the sample. These high-frequency spectra are effectively shifted by a single spatial frequency in the grid pattern and are passed through the cutoff frequency of the imaging system, and they overlap the low spatial frequency spectra near the DC frequency, permitting small structures in the sample to be observed, as illustrated in Fig. 1(c). These overlapped spectra can be extracted by optical sectioning, as shown in Fig. 1(d). After optical sectioning, the cutoff frequency of the reconstructed frequency spectrum is extended by the spatial frequency (${\mathbf{k}}_{\mathbf{0}}$). Therefore, the cutoff frequency is higher than that of a conventional optical microscope, and the high-frequency spectra of the sample can be observed. In particular, optimal sectioning occurs when the spatial frequency (${\mathbf{k}}_{\mathbf{0}}$) is half of the cutoff frequency (${\mathbf{k}}_c$) of the OTF.^2,13

To cover the entire frequency spectrum of the sample, the OTF must be effectively extended in multiple orientations by rotating the grid pattern. A description of the raw image, with a frequency spectrum of the sample by the OTF extended to the entire area, can be written as follows:

The rotation shows different areas in the frequency spectrum of the sample that can provide more accurate height information of the sample and reduce the background noise. However, high-frequency spectra overlap with the low-frequency spectra, which degrades the raw image quality before optical sectioning. Therefore, these components must be extracted and reconstructed for observing small structures in the sample. To separate them, optical sectioning is performed by the square law detection from three phase-shifted images, which correspond to the relative spatial phases of ${\varphi }_{\mathrm{0,1},\mathrm{2,3}}=0,{\varphi }_{\mathrm{0,1},\mathrm{2,3}}=2\pi /3,{\varphi }_{\mathrm{0,1},\mathrm{2,3}}=-2\pi /3$ After optical sections in multiple orientations, the reconstructed frequency spectrum of the sample extended to other frequency spectrum regions, as depicted in Fig. 1(e). Four optical sections corresponding to four different orientations of the grid pattern are required at any one longitudinal position to measure the entire frequency spectrum of the sample. After optical sectioning, the components in which the high-frequency spectrum overlaps the low-frequency spectrum are extracted, and the low-frequency components are eliminated. As a result, we can obtain the sectioned image with the entire frequency spectrum of the sample. It is expected to provide more accurate height information and reduce background noise.

3.1.

Experimental Setup

An optical imaging system was constructed to operate the SIM, as shown in Fig. 2. The warm white LED (Thorlabs, MWWHLP1) was used as the light source; the grid pattern loaded on the SLM was projected onto the sample through the objective lens (Mitutoyo, 20X Plan Apo Infinity Corrected Long WD Objective); and the final image with the grid pattern was formed on the camera (The Imaging Source, DMK 33GX174e). The image of the sample was clearly formed on the camera when the grid pattern was matched on the sample. The phase shift was implemented by electrically shifting a sinusoidal wave pattern by one third of the spatial period of the pattern in the SLM, which has a spatial period of $0.83\mu \mathrm{m}$ and a pixel pitch of $0.003\mu \mathrm{m}$ along with the objective lens used in the experiment. Optical sectioning was performed by square-law detection using three phase-shifted images with zero spatial phase. As the sample was axially scanned, we took optically sectioned images for all axial positions and extracted the height information of the sample at the focused axial position. Finally, a 3-D height map was reconstructed based on these sectioned images.

Fig. 2

(a) Optical configuration of SLM-based SIM; L, lens; BS, beam splitter; BPF, band-pass filter; OL, objective lens; and (b) experimental setup.

3.2.

Designing the Grid Pattern in the SLM

As the spacing of the grid pattern becomes narrower, the normalized spatial frequency becomes close to unity. Hence, the optical sectioning strength becomes stronger than that of the low spatial frequency of the grid pattern designed for the SLM. In the experiment, we designed grid patterns with different spatial frequencies ($v$) that were 100 line pairs/mm, 64 line pairs/mm, and 50 line pairs/mm, and the normalized spatial frequencies were 1.16, 0.75, and 0.58, respectively, by calculating $\tilde{v}=\beta \lambda v/\mathrm{NA}$, where $\beta$ and $\lambda$ denote the magnification of the imaging system and the wavelength, respectively, $v$ is the spatial frequency of the grid pattern and NA is the numerical aperture. Raw images in grid patterns with different spatial frequencies were formed, as shown in Fig. 3. Of course, all the methods presented in this paper are based on the invariant PSF. In a noninvariant PSF, the axial extent of the PSF increases as the focal position becomes deeper along the $Z$-axis, which causes aberration.^14,15 However, the objective lens of our system was well calibrated, and aberrations did not occur because each relay lens was well placed at the focal distance. Figure 3 shows that there was no distortion or blurring at the edge of the grid pattern, which indicates that aberration did not occur. The axial responses between each system were measured under the condition that the system was stable.

Fig. 3

Designing the grid pattern with spatial periods of (a) $10\mu \mathrm{m}$, (b) $15.63\mu \mathrm{m}$, and (c) $20\mu \mathrm{m}$ in the SLM.

Axial responses were measured for the grid patterns with different spatial frequencies, as shown in Fig. 4, to investigate the correlation of the spatial frequency with the optical sectioning strength. We introduce the FWHM to facilitate the comparison of these axis responses. The FWHM refers to actual defocus at an axis response of 0.5. In general, normalized spatial frequency and normalized defocus are inversely proportional, so the higher the normalized spatial frequency, the lower the normalized defocus. As the normalized defocus $u$ is related to the actual defocus $z$ through $u=8(\pi /\lambda )z{\sin }^2(\alpha /2)$, we can estimate the actual defocus by obtaining the normalized defocus according to the normalized spatial frequency, and comparing actual defocuses can evaluate each axis response.² Therefore, it can be confirmed from the aforementioned equation that the actual defocus changes by the normalized spatial frequency, which affects the axial response. The FWHMs of axial responses at each normalized spatial frequency were measured to facilitate comparison of axial responses. In practice, we measured 15.8, 18.8, and $20\mu \mathrm{m}$ for the FWHMs of axial responses, which correspond to grid patterns with spatial periods of 10, 15.63, and $20\mu \mathrm{m}$, respectively. The maximum optical sectioning strength was shown at the grid pattern with the spatial period of $10\mu \mathrm{m}$ in which the normalized spatial frequency is close to unity. As the FWHM narrowed, there was an improvement in the axial response, and the FWHM was minimized when the normalized spatial frequency was approaching unity, as shown in Fig. 4. Consequently, the FWHM continuously became narrower as the normalized spatial frequency approached unity, leading to improved axial response. For the grid pattern with the narrowest FWHM of the axial response, SIM provides high precision when extracting the height information of the sample to be observed.

Fig. 4

Measured axial responses of the system.

3.3.

Reconstructing the Sectioning Image

We performed SIM experiments with a coin as the sample to verify the effects of different axial responses on SIM performance. Figure 5 shows the sectioning image, which is reconstructed by a square-law detection algorithm from the phase shift images. First, we discuss the sectioned image in the out-of-focus region. In this region, the sectioned image, which is obtained by Eq. (1), should have zero intensity for all image pixels, provided that the object is ideally out-of-focus. However, as shown in Figs. 5(b) and 5(c), the sectioned images of the object are constructed even in the out-of-focus area. This means that the image is measured as if there are samples even in the out-of-focus image, causing an error in the 3-D height map information. In addition, the shorter period of the grid pattern of the SLM leads to the smaller intensity of the sectioned image relative to the larger period of the grid pattern in out-of-focus. This is because the depth-of-field becomes smaller as the period becomes smaller. It is also consistent with the narrower FWHM discussed in Sec. 3.2. This means that the imaging system has a high dynamic range in a short spatial period of the grid pattern and can observe the sample well in the focal plane. Consequently, it is expected that the smaller period of the grid pattern provides more accurate height map information. Second, we discuss the sectioned image in the focused area. In this area, as shown in Figs. 5(d)–5(f), the shorter period results in significantly clearer surface information in the sectioned image. As the period of the grid was sufficiently shortened, the sectioning strength was improved, which is described in Sec. 2. Consequently, it is expected that the sufficient small period of the grid pattern provides more accurate height map information (Fig. 6).

Fig. 5

Sectioned images of the coin in the out-of-focus region using grid patterns with spatial periods of (a) $10\mu \mathrm{m}$, (b) $15.63\mu \mathrm{m}$, (c) $20\mu \mathrm{m}$, and in the in-focus region using grid patterns with spatial periods of (d) $10\mu \mathrm{m}$, (e) $15.63\mu \mathrm{m}$, and (f) $20\mu \mathrm{m}$.

Fig. 6

Frequency spectra of the grid in the (a) ${\mathbf{k}}_{\mathbf{x}}$ direction and the (b) ${\mathbf{k}}_{\mathbf{y}}$ direction by effectively extended OTF.

We also rotated the grid pattern in multiple orientations and merged each sectioned image to obtain the sectioned image of the fully extended OTF in all areas. This showed that the OTF was shifted in the ${\mathbf{k}}_x$ or ${\mathbf{k}}_{\mathbf{y}}$ direction when optical sectioning was performed in only one orientation, whereas the OTF was effectively expanded in the ${\mathbf{k}}_{\mathbf{x}}$ and ${\mathbf{k}}_{\mathbf{y}}$ directions when optical sectioning was performed in multiple orientations. It was demonstrated that the fully extended frequency spectra of the sample were observed in the raw image with the rotated grid pattern. From the results, the axial resolution was increased in the grid pattern with the normalized spatial frequency of unity compared with other spatial frequencies (Fig. 7).

Fig. 7

Height maps of the plane mirror reconstructed by grid patterns with spatial periods of (a) $10\mu \mathrm{m}$, (b) $15.63\mu \mathrm{m}$, and (c) $20\mu \mathrm{m}$.

3.4.

Reconstructing the Height Map

The 3-D height maps were reconstructed for the plane mirror between each system. The axial scanning was performed with $1\text{-}\mu \mathrm{m}$ step and total scan range was $100\mu \mathrm{m}$. A better optical sectioning occurred in the grid pattern with the spatial period of $10\mu \mathrm{m}$, where the normalized spatial frequency is close to unity. It was confirmed that the height map reconstructed by the grid pattern with the spatial period of $10\mu \mathrm{m}$ had more accurate height information compared with height maps reconstructed by grid patterns with spatial periods of 15.625 and $20\mu \mathrm{m}$.

To examine the roughness of the plane mirror, standard deviations were calculated for the reconstructed 3-D height maps. The standard deviation values were 0.25, 0.37, and $0.39\mu \mathrm{m}$, which correspond to grid patterns with spatial periods of 10, 15.625, and $20\mu \mathrm{m}$, respectively. This showed that the height map reconstructed by the grid pattern with the normalized spatial frequency of unity yielded a smaller standard deviation compared with height maps reconstructed by grid patterns with different spatial frequencies. These results suggest that SIM can provide a high precision measurement for grid patterns with normalized spatial frequency of unity (Fig. 8).

Fig. 8

Height maps of the coin reconstructed by grid patterns with spatial periods of (a) $10\mu \mathrm{m}$, (b) $15.63\mu \mathrm{m}$, and (c) $20\mu \mathrm{m}$.

We also measured the height of the coin for a region of interest (ROI) of size $100\times 100{\mu \mathrm{m}}^2$ in the 3-D height map. To obtain the objective data, we set the value measured with the confocal microscope (PRECITEC, CHRocodile C) as the reference value. Each optical sectioning was evaluated by comparing the reference value to the value measured by grid patterns with different spatial frequencies. The axial scanning was performed with $0.7\text{-}\mu \mathrm{m}$ step and a scan range of $700\mu \mathrm{m}$. The height was computed by subtracting the average height of the two faces of the ROI. The height of the coin in the ROI was measured as 29.44, 30.96, and $30.8\mu \mathrm{m}$, which correspond to grid patterns with spatial periods of 10, 15.625, and $20\mu \mathrm{m}$, respectively. The height of the same ROI was also measured as $30\mu \mathrm{m}$ using the confocal microscope. The SIM using the grid pattern with an appropriately high spatial frequency provided high accuracy, considering that the error in the measurement of the confocal microscope is 5%.

For optical sectioning performed in multiple orientations, the noise was dramatically reduced in the height map, as shown in Fig. 9. The grid pattern was rotated by 45 deg. At least three rotations (four grid orientations) allowed for the measurement of the entire frequency spectrum of the sample. The frequency spectrum of the sample was measured by an effectively extended OTF. The more the area occupied by the OTF, the more uniformly the frequency spectrum of the sample can be measured. Therefore, if the rotation angle of the grid pattern is reduced and rotated several times, more frequency spectrum areas of the sample can be uniformly measured. Of course, measuring multiple rotations increases the integration time of optical sectioning images, which is disadvantageous for the measurement speed. The number of grid orientations is equal to 180 divided by the angular step size. Considering the measurement speed, we adopted an angular step size of 45 deg. In other words, the grid pattern was rotated three times, and the number of grid orientations was 4. This provided adequate measurement speed and uniform frequency spectrum measurement. It is shown in Fig. 9(b) that noise was reduced in the 3-D height map.

Fig. 9

Height maps of the coin in the (a) original view using single sectioning and the (b) original view using multiple sectioning.

Therefore, we successfully optimized the SIM by designing a proper grid pattern and performing optical sectioning in multiple orientations in the SLM. The accuracy obtained from the measurement is comparable to that of a confocal microscope.