1.

Introduction

Three-dimensional (3-D) as well as two-dimensional (2-D) optical and image processing techniques have been investigated to identify specific objects in unknown scenes. ^{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13} Automatic and real-time identification of microorganisms has vast potential for various applications such as detection of biological weapons and harmful diseases, diagnosis of diseases, investigation of food safety, and ecological monitoring. There are conventional methods to identify microorganisms; however, the conventional techniques require time-consuming culturing and biochemical analysis with special skills. Therefore, automated and real-time recognition of microorganisms using 3-D optoelectronic imaging and can be beneficial.

Automated discrimination of living microorganisms in unknown images is very challenging. Tiny biological living objects can have simple and undistinguishable morphologies between different species; and many morphological variants exist in the same class.¹⁴ Research and development in this field have been performed using specific color and shapes based on captured 2-D intensity images.^{15, 16, 17, 18, 19, 20} The identification of tuberculosis bacteria and Vibrio cholerae has been studied based on their colors and 2-D shapes.^{15, 16} In Ref. 17, bacteria in a wastewater treatment plant are identified by morphological descriptors. The aggregation of streptomyces is classified into different phases by measuring the aggregation size and reaction time.¹⁸ In Ref. 19, plankton recognition is performed using preselected geometrical features. More research on image analysis and recognition of microorganisms can be found in Ref. 20. Recently, 3-D microorganism recognition was proposed using the single-exposure on-line (SEOL) digital holography.^{3, 13}

In this paper, complex information (magnitudes and phase) is utilized, providing distinct features that are impossible to be observed on 2-D intensity images. The phase change is due to the retardation of light as it propagates through the biological sample. We propose the automatic selection process of training data and present experimental results on the recognition of microorganisms. Figure 1 shows the block diagram of the recognition system. Microorganism in the Fresnel diffraction field is recorded by SEOL digital holography.^{11, 12, 13} Complex amplitude holographic images^{21, 22, 23, 24} are reconstructed at arbitrary depths by the inverse Fresnel transformation. The complex morphological pattern is segmented and salient features are extracted by the Gabor-based wavelets.^{25, 26} The training data is automatically selected by means of Gabor feature vectors and the graph matching technique.^{27, 28, 29, 30, 31} The automatic selection of training data is useful when biological samples overlap and∕or cluster, which make it difficult to select individual objects as training data. The rigid graph matching (RGM) techniques measures the similarity of complex geometrical shapes between a reference microorganism and unknown biological objects. A training rule is applied and the mean vector is stored into the database for the known reference. For the identification of unknown inputs, Euclidean distance between the reference mean vector and the feature vector of input objects are compared with a threshold.

Fig. 1

Block diagram of the automatic and real-time identification system.

Section 2 briefly reviews the recording and reconstruction of the SEOL digital holography. The segmentation and Gabor-based wavelets are illustrated in Sec. 3. Section 4 describes the feature vector selection with graph matching. Section 5 presents the training and decision processes. The experimental results and conclusions follow in Secs. 6, 7, respectively.

2.

Review of SEOL Digital Holography

In this section, we review the recording and reconstruction of the 3-D complex information. The complex wave in the Fresnel diffraction field is recorded by the microscope-based Mach-Zehnder interferometer [see Fig. 2a ]. The 3-D complex holographic images are computationally reconstructed by the inverse Fresnel transformation at arbitrary depth $\left(d\right)$ [see Fig. 2b]. The SEOL digital holography is adopted for its advantages such as real-time detection and robustness to environmental fluctuation.¹³ The reconstruction process of the SEOL digital holography is described in Appendix A. Figure 3 shows the magnitude and the phase of the reconstructed holographic image of Sphacelaria alga.

Fig. 2

Schematic diagrams for (a) optical setup of SEOL digital holography and (b) the reconstruction process at arbitrary depth.

Fig. 3

Computationally reconstructed holographic image of Sphacelaria alga $(d=180\mathrm{mm})$ for (a) magnitude and (b) phase.

3.

Segmentation and Feature Extraction

Since the coherent light is scattered by the semitransparent objects, the intensity on the foreground objects is lower than the background field. During the preprocessing, we subtract the background diffraction field in the reconstructed images. The detailed segmentation process is presented in Appendix B.

After the segmentation, features of microorganisms are extracted by the Gabor-based wavelets to generate a node vector at each pixel. Gabor-based wavelets are composed of multioriented and multiscaled Gaussian-form kernels that are suitable for local spectral analysis.^{25, 26} We define a node vector at the pixel $(m,n)$ as

4.

Automatic Training Selection Using Graph Matching

In this section, we utilize the graph-matching technique to choose the data for the training process. The graph-matching technique was developed for pattern recognition of distorted objects.^{27, 28, 29, 30, 31} However, in the case that the image pattern of a reference object cannot be individually observed, or they are clustered as may be the case with some microorganisms, the selection of training data can be a labor-intensive task. In this paper, we achieve the fully automated recognition system by choosing the feature vectors to be trained by means of the RGM technique.

Let $R$ and $T$ be two identical and rigid graphs that are defined as sets of nodes associated in the local area. The graph $R$ and $T$ are placed on the image $O_r$ and the image $O_t$ of the same reference microorganism, respectively. The reference graph $R$ is translated by a fixed translation vector ${\mathbf{p}}_r$ and rotated by a fixed clock-wise rotation angle ${\theta }_r$ to cover a predetermined a referenced morphology. Therefore, the position vectors of the nodes in the reference graph $R$ are computed as

We sequentially search for a similar local morphology with the referenced morphology on the image $O_r$ by translating and rotating the graph $T$ on the image $O_t$ . We choose the node vectors ${\mathbf{v}}_T\left[{\mathbf{x}}_k({\mathbf{p}}_t,{\theta }_t)\right]$ [see. Eq. 1] of the graph $T$ as training data if the following two conditions are satisfied:

The similarity function and the difference cost in Eq. 6 are defined as

5.

Training and Decision Process

We define a feature vector as the collection of the node vectors of the graph. The feature vector to be trained is presented as

To recognize unknown inputs, the graph $S$ is translated and rotated on the unknown image $O_s$ as in the training procedure [see Eq. 5]. The graph $S$ has the same shape and size with the graph $R$ and $T$ . We compare the Euclidean distance with a threshold at every translation vector ${\mathbf{p}}_s$ and the rotation angle ${\widehat{\theta }}_s$ . We accept the detection of the referenced 3-D complex morphology if the following condition is satisfied:

6.

Experimental Results

In the recording of the SEOL digital hologram,¹³ the size of the CCD for SEOL digital hologram is $2048\times 2048\text{pixels}$ and $1\text{pixel}$ size is $9\times 9\mu \mathrm{m}$ . The CCD is placed $500\mathrm{mm}$ from the microorganism samples. The thickness of the microorganisms varies between 10 and $50\mu \mathrm{m}$ . To test the recognition performance under various circumstances, we generate SEOL holograms of nine Sphacelaria alga and Tribonema aequale alga samples, respectively. We denote nine Sphacelaria samples as A1,…,A9 and nine Tribonema aequale samples as B1,…,B9. Since we have changed the position of the CCD during the experiments to test the robustness of the recognition system, the reconstruction depth for the focused image varies from $180\text{to}300\mathrm{mm}$ . After the reconstruction, the magnitude and phase information of computationally reconstructed holographic 3-D images are cropped and reduced into $256\text{‐}\times 256\text{‐}\text{pixel}$ images with a reduction ratio of 0.25 considering the computational complexity.

For the segmentation, we assume less than 20% of lower intensity region is occupied by microorganisms and the intensity of microorganisms is less than 45% of the background diffraction field. Therefore, the maximum intensity rate $r_{\max }$ in Eq. 18 and the segmentation probability $P_s$ in Eq. 19 are set at 0.45 and 0.2, respectively. The parameters for Gabor-based wavelets are set at $\sigma =\pi$ , $k_0=\pi ∕2$ , $\delta =\sqrt{2}$ , $U=5$ , and $V=6$ in Appendix C. Figure 4 shows the component of the node vectors in Eq. 1.

Fig. 4

Components of the node vectors when (a) $u=1$ , (b) $u=2$ , (c) $u=3$ , (d) $u=4$ , and (e) $u=5$ .

A rectangular grid is selected as a reference graph for the Sphacelaria alga, which shows regular thickness (see Fig. 3). The reference graph $R$ is composed of $25\times 3$ nodes and the edge distance between nodes is $4\text{pixels}$ in the $x$ and $y$ directions. Therefore, the total number of nodes in the graph is 75. The reference graph $R$ is placed with ${\mathbf{p}}_r={[81,75]}^t$ and ${\theta }_r=135\mathrm{deg}$ in the sample image A1. Considering the computational load, the graphs $T$ is translated by every $3\text{pixels}$ in the $x$ and $y$ directions for measuring the similarity and difference to the graph $R$ for the training data selection. To search the best matching angles, the graph $T$ is rotated by $7.5\mathrm{deg}$ from $0\text{to}180\mathrm{deg}$ at every translated location. When the positions of rotated nodes are not integers, they are replaced with the nearest neighbor nodes. For the training data selection, the thresholds ${\alpha }_S$ and ${\alpha }_C$ are set at 0.95 and 1.2, respectively, in Eq. 6. For the decision process the threshold ${\alpha }_D$ is set at 9.5 in Eq. 13. Figure 5a shows the reference graph on the sample image A1 and Fig. 5b shows automated selection of the training data where 33 graphs (feature vectors) are selected. Figures 5c and 5d show examples of recognition results performed on the true class sample A3, where 159 feature vectors are identified, and the false class sample B1, where no detection is accepted. Figure 6a shows the number of detection in true class samples A1 to A9 and false class samples B1 to B9. The number of detection varies from 14 to 159 in true class samples A1 to A9. No detection is accepted in false class samples B1 to B9. Figure 6b shows the minimum Euclidean distance in all samples. The minimum Euclidean distances in false class samples B1 to B9 are larger than those in true class samples A1 to A9 showing the discrimination capability of the proposed recognition system.

Fig. 5

(a) Reference graph on the sample A1, (b) training data selection on the sample A1, (c) decision result on true class sample A3, and (d) decision result on false class sample B1.

Fig. 6

(a) Number of detections and (b) minimum Euclidean distance.

7.

Conclusions

In this paper, we have described the identification of biological microorganisms using their 3-D complex amplitude of geometrical information obtained by computer-reconstructed SEOL holographic images. ^{14, 32} Segmentation, Gabor feature extraction, feature vector selection using graph matching, and training and decision processes are presented to recognize a predetermined 3-D morphology in unknown biological samples. We choose the feature vectors to be trained by means of the RGM technique to remedy the possibility of biological samples overlap and∕or cluster, which make it difficult to select individual objects for the training purpose. Training and decision rules are applied to show the performance of the proposed system. Note that more sophisticated training and decision rules can be considered,^{7, 19} depending on the kind of microorganisms.