1.

Introduction

Medical rigid endoscopes (MREs) consist of three parts of optical lenses: the objective lens, the relay system, and the eyepiece. Great efforts for technological advancements in optical lenses and structural optimization have brought direct conveniences to their clinical applications. Therefore, the imaging quality directly affects the realization of more applications of MREs. As for an MRE, aimed to inspect as large an area as possible within the patient body under the least movement for routine testing, early cancer screening, and clinical surgery through small holes in a non-invasive or minimally invasive way, it is deliberately designed to have a wide field of view(FOV). The wider FOV makes the magnification no longer constant, so the output image loses the similarity with the input target and thus presents geometric distortion, reducing the accuracy of modulation transfer function (MTF) evaluation. In addition, the wide FOV presents images with non-uniform illumination distribution and field curvature. The non-uniform illumination distribution contributes to slight difference on the MTF evaluation before and after illumination compensation.¹ Field curvature can cause the image to be blurred, because field curvature cannot make the center and edges of the FOV clear at the same time. Therefore, by focusing on the center of the FOV, blurring can be avoided to be introduced into its MTF evaluation. Thus, the geometric distortion is a major problem in MTF evaluation for optical lenses with poorly corrected geometric distortion. Until recent years, some research work has been done. Masaoka et al.² used a modified slanted-edge method to evaluate the MTF for the fisheye lens by ISO Standard 12233. Williams and Burns³ used a slanted edge gradient SFR method to measure the MTF of the distorted edge images. Yet, the study between the geometric distortion and the sampling region of interest (ROI) image is seldom involved. It may cause the MTF evaluation to be affected by the inappropriate sampling ROI images, reducing the accuracy. Thus in this paper, a based liquid-crystal-device MTF (LMTF) method is proposed for this purpose. This method utilizes two computer-generated targets to establish a mathematical model of geometric distortion and then analyzes two sampling ROI image manners in MTF evaluation. Under the limitation of the sampling ROI image based on the distortion factor (DF) manner, the sampling ROI image based on the distortion correction manner is presented for the MTF evaluation of the MRE. One target is characterized with a visual uniform structure, which is responsible for establishing a mathematical model of geometric distortion. A Hadamard target, created by a pseudorandom Hadamard generator, has a natural geometric center, which helps to save time for the experimental system in alignment adjustment. The Hadamard target is derived from a Hadamard matrix $H_n$, which is a square matrix with the order $n$ equaling to $2^k$ where $k$ is a positive integer. The other target is characterized with a uniform spectral distribution for the MTF evaluation. The spectral distribution can be expressed as a uniform contrast, a uniform spatial frequency spectral density, or a uniform spatial power spectral density. Targets of such spectral distribution are also widely used for the MTF evaluation of the optical lenses with well-corrected distortion.^4–15 Here, the target with a uniform power spectral density is mainly considered, and it can be created by a random number generator.¹⁶ The generated random target is characterized with the random numerical distribution and verify the LMTF method.

The LMTF method presents a feasible solution for the MTF evaluation of the MRE. In addition, the LMTF method also presents advantages by utilizing the liquid crystal device (LCD) to present digital targets in equivalent pixels. First, it only needs to complete one alignment adjustment and optimization for the experimental system, thereby reducing the risk of errors caused by frequent alignment movement into the experimental results. Second, it can visually observe the clarity of the target image in the central FOV, avoiding the blur caused by field curvature into the central FOV, thereby inhibiting the error caused by field curvature from entering the experimental results. Third, it more flexibly displays targets at different quasi-monochromatic illumination by the LCD itself than printed targets, and the MTF can be evaluated at different wavelengths.

2.

Experimental Setup and MTF Analysis

2.1.

Experimental Setup

The LCD is mounted on a mobile platform operated in multiple directions, and its distance from the entrance pupil of the MRE is $L_0$. The black and white CMOS camera has $1280\times 1024$ image pixels with equal pixel pitch $d_{\mathrm{CMOS}}$ of $5.2\mu \mathrm{m}$. The optical interface (OI) connecting the MRE and the CMOS camera is used for converting the virtual image into the real image. The experimental setup is shown in Fig. 1.

Fig. 1

Experimental setup.

The MTF of the OI is high enough to prevent its MTF from affecting the evaluated MRE. The Nyquist spatial frequency $f_N$ of the CMOS camera is determined by its pixel pitch and is equal to $1/(2\times d_{\mathrm{CMOS}})$. The highest spatial frequency $f_H$ of the target image onto the CMOS camera is equal to $1/(2\times d_{\mathrm{LCD}}\times M)$, where $d_{\mathrm{LCD}}$ is the pixel pitch of the LCD and $M$ is the magnification of the experimental system. To avoid frequency aliasing, that is, to prevent the spatial frequencies higher than the Nyquist spatial frequency from being mapped to the lower spatial frequencies of the image to affect the experimental results, $f_H\le f_N$ is established. Further, it can be derived as

Eq. (1)

$$M\ge d_{\mathrm{CMOS}}/d_{\mathrm{LCD}},$$

where $d_{\mathrm{LCD}}$ is an important parameter to ensure successful experimental implementation, that is, different LCDs with the same $d_{\mathrm{LCD}}$ achieve the same experimental measurement. As described in Eq. (1), an oversampling mode is often used to extract the true law contained in the raw data. However, as $M$ moves away from the ratio of $d_{\mathrm{CMOS}}/d_{\mathrm{LCD}}$, more duplicate data, rather than more valid data, are sampled. Overemphasizing the amount of data can lead to overfitting of experimental results. Therefore, $M$ close to the ratio of $d_{\mathrm{CMOS}}/d_{\mathrm{LCD}}$ is a more preferred choice. Here, the LCD has a 5.1-in. screen with $1920\times 1080$ image pixels, and then its $d_{\mathrm{LCD}}$ is equal to $58.8\mu \mathrm{m}$. So the value of the minimum $M$ ($M_{\min }$) is equal to 0.09. To fully establish the mathematical model of geometric distortion in the entire FOV, the matrix size of the Hadamard target is not less than $(L\times W)\times {(d_{\mathrm{CMOS}}/(d_{\mathrm{LCD}}\times M_{\mathrm{ave}}))}^2\text{pixels}$. Here, $L$ and $W$ are the horizontal pixel length and vertical pixel width of the target image, respectively, and $M_{\mathrm{ave}}$ is the average magnification,

Eq. (2)

$$M_{\mathrm{ave}}=\frac{1}{r}\int M\mathrm{d}r.$$

Therefore, by calculation, the matrix size of the Hadamard target is not less than $512\times 512\text{pixels}$.

3.

Mathematical Model of Geometric Distortion

3.1.

Sampling ROI Image Determined by the Distortion Factor Manner

In Fig. 2, a mapping relationship between the object and image planes is established, where both center points are marked in green. A coordinate system is used, where $r^{\prime }$ means the pixel distance from the center to the other points of the image plane and $r$ means the pixel distance from the center to the corresponding mapped points of the object plane. The furthest pixel distances for $r^{\prime }$ and $r$ are both normalized to 1. The Hadamard image is axisymmetric; $r^{\prime }$ can be measured from the center to any direction, for example, from the center to the right or upper direction. Thus, the coordinates of the selected six points (in pink or blue) are read, and the pixel distances from the center to them are calculated and normalized. As for $r$, the mapped points in the Hadamard target are adjacently equally spaced. After taking the data of $r^{\prime }$ and $r$, and performing a nonlinear least-squares method, a fitting polynomial is written to describe the relationship between $r^{\prime }$ and $r$,²¹

Eq. (7)

$${r^{\prime }}_{\text{fitting}}=k_m{r}^m+k_{m-1}{r}^{m-1}+\dots +k_2{r}^2+k_1{r}^1+k_0,$$

where $m$ is the order of the polynomial equation, $r^{\prime }\in [\mathrm{0,1}]$, and $r\in [\mathrm{0,1}]$. The distortion rate function $g^{\prime }(r)$ is to quantify the degree how the MRE distorts the image and then is the derivative of Eq. (7),

Eq. (8)

$$g^{\prime }(r)=\frac{{d{r}^{\prime }}_{\text{fitting}}}{dr}=m{k}_m{r}^{m-1}+(m-1)k_{m-1}{r}^{m-2}+\dots +2k_{2}r+k_1,$$

where $g^{\prime }(r)$ also stands for the magnification function since $r^{\prime }$ and $r$ represent dimensionless length information. For an ideal distortion-free imaging system, $g^{\prime }(r)$ is equal to 1 [black line in Fig. 3(a)], which means that the output image is proportional to the target. Then for an actual imaging system such as the MRE, the output image is distorted to magnify the target [pink and blue curves in Fig. 3(a)]. The closer to the edge, the more pronounced the distortion. Therefore, the subtraction of the non-distorted $g(r)$ and the distorted $g^{\prime }(r)$ is

Eq. (9)

$$\mathrm{\Delta }{g}^{\prime }(r)=g^{\prime }(r{)}_{\text{non-distorted}}-g^{\prime }(r{)}_{\text{distorted}},$$

where $\mathrm{\Delta }{g}^{\prime }(r)$ indicates how the magnification with distortion deviates from that without distortion. Since the distortion is distributed along the radial direction, the $\mathrm{\Delta }{g}^{\prime }(r)$ can be used to quantify the degree of distortion, and the value of $r$ can be used to describe the DF value. Therefore, the DF value can be directly related to the size of the sampling ROI image, and the maximum size of the valid sampling ROI image is 1, where DF value is equal to $\alpha$. Here, $\alpha$ is the maximum non-distorted factor, that is, the geometric distortion of the relevant sampling ROI image is negligible, and it corresponds to 0.05 for both directions based on the slope of the local magnification in Fig. 3(b) where the local magnification has a slope close to zero. The horizontal pixel length and vertical pixel width for the valid ROI image are equal to $\alpha \times L$ and $\alpha \times W$, respectively, where $L$ and $W$ are both equal to 891. Then the matrix size for the valid sampling ROI ranges from $2f_{\max }\times 2f_{\max }$ to ${\alpha }^2\times L\times W$, where ${\alpha }^2\times L\times W$ represents the maximum matrix size of the valid sampling ROI image, that is, the matrix range for the valid sampling ROI image is from $33\times 33$ to $44\times 44$. Then the valid sampling ROI image can be approximated as an imaging region with uniform magnification, and its data process can refer to a linear procedure. As for the magnification function of the experimental system shown in Fig. 3(c), it is written as

Eq. (10)

$$M(r)=c*g^{\prime }{(r)}_{\text{distorted}},$$

where $c$ is the magnification value of the valid sampling ROI image, and its value of 0.19 is greater than $M_{\min }$ so as to avoid the frequency aliasing for the experimental results.

Fig. 2

Target and image: (a) Hadamard target and (b) Hadamard image.

Fig. 3

Sampling ROI image determined by the DF manner: (a) $g^{\prime }(r)$, (b) $\mathrm{\Delta }{g}^{\prime }(r)$, and (c) $M(r)$.

3.2.

Extended Sampling ROI Image by Distortion Correction Manner

As mentioned above, geometric distortion directly leads to non-uniform magnification, compromising the uniformity of the sampling ROI image. So to increase accuracy of the MTF evaluation, the size of the valid sampling ROI image is limited within the size determined by the maximum non-distorted factor $\alpha$. To obtain an extended uniform sampling ROI image, the geometric distortion correction manner is performed on the target image to compensate for the attenuation of magnification. Therefore, by dividing the function $g^{\prime }(r)$, the corrected image can be written as follows:

Eq. (11)

$${\text{image}}_{i^{\prime }{j}^{\prime },\text{corrected}}={{\text{image}}_{ij,\text{original}}/g^{\prime }(r)|}_{r\in \mathrm{ROI}},$$

where $ij$ stands for the meridional and sagittal coordinates in the original image, and $i^{\prime }{j}^{\prime }$ represents the coordinates in the corrected image. It is worth noting that a pixel coordinate point of the corrected image may correspond to the position among several coordinate points on the target image. Therefore, grayscale interpolation is required to calculate the grayscale value of the corrected pixel coordinate points. To keep the frequency content of the corrected image as much as possible, the nearest neighbor interpolation algorithm is used in the distortion correction manner to resample the grayscale values of the coordinate points in the corrected image shown in Fig. 4. Then base on Eq. (11), the size of the valid sampling ROI image can be extended for the MTF evaluation.

Fig. 4

Distortion correction manner: (a) corrected Hadamard image and (b) corrected random image.

4.

MTF Results

The difference between the evaluated MTF and the theoretical MTF is used as an evaluating index. This index describes the relative root mean square error (RMSE) of the MTF evaluation,

As presented in Fig. 5, the analysis between the DF and the size of the sampling ROI image shows the relationship between geometric distortion and MTF evaluation. As DF value is no more than $\alpha$, that is, the size of the valid sampling ROI images within the range of $33\times 33$ and $44\times 44$ such as $36\times 36$, $40\times 40$, and $44\times 44$, their MTF curves are close to each other. Then the MTF curves are compared with that of the theoretical prediction, their $R_{\mathrm{RMSE}}$ values are 6.5%, 7.3%, and 7.6% (7.2% on average), respectively. As DF value is between $\alpha$ and $2\times \alpha$, that is, the matrix size of the sampling ROI image within the range of $44\times 44$ and $90\times 90$, their MTF curves obviously drop at lower frequencies, and their $R_{\mathrm{RMSE}}$ values rise to 8.2% to 11.7% (10.1% on average). Thus as the DF value continues to increase, the $R_{\mathrm{RMSE}}$ value also continues to rise. For example, as DF value exceeds $2.5\times \alpha$, the $R_{\mathrm{RMSE}}$ value is more than 14.7%. As the DF value reaches $10\times \alpha$, the corresponding MTF curve deteriorates drastically. This shows that DF value has a significant impact on the accuracy of the MTF evaluation. Therefore, the size of the valid sampling ROI image based on the DF manner is limited to the size, where DF value is equal to $\alpha$.

Fig. 5

MTF results based on the DF manner: (a) the random image and the selected ROI image and (b) evaluated MTFs comparison with the theoretical MTF.

As the distortion correction manner is applied to the target image, the results of the MTF evaluation based on the corrected ROI images are shown in Fig. 6. As DF value is no more than $\alpha$, the sampling ROI images such as $36\times 36$, $40\times 40$, and $44\times 44$, their $R_{\mathrm{RMSE}}$ values drop to 3.8%, 3.9%, and 4.2% (4% on average), respectively. The results indicate that the MTF curves are further optimized compared with the DF manner. As DF value is between $\alpha$ and $2\times \alpha$, the MTF curves are improved at low frequencies compared with those corresponding curves based on the DF manner. The average $R_{\mathrm{RMSE}}$ value drops to 6%, which means an average 40% reduction occurs. The results indicate that the distortion correction manner is superior to the DF manner in which the average $R_{\mathrm{RMSE}}$ value is reduced from 10.1% to 6%, which is an obvious improvement compared with 7.2% based on the maximum non-DF $\alpha$. Therefore, a 1.2% improvement of accuracy reflects that the distortion correction manner relaxes the DF value to $2\times \alpha$, extends the sampling ROI image to twice that of the DF manner, and reduces the average $R_{\mathrm{RMSE}}$ value to 6%. Therefore, the size of the valid sampling ROI image based on the distortion correction manner can extend to twice the size of the valid sampling ROI image based on the DF manner. Even as DF value increases to $2.5\times \alpha$, the $R_{\mathrm{RMSE}}$ value drops to 11.4%, thus optimized by 22%. However, as the DF value exceeds $3\times \alpha$, that is, the size of the sampling ROI image extends outward from the central FOV, the blur caused by field curvature begins to be included in the sampling ROI image. Due to field curvature, the sharpness of the image to gradually change from the central FOV to the edges, and the distortion correction manner helps the blur distribution caused by the field curvature to be more uniform. This leads to a more prominent influence of field curvature on the results of the MTF evaluation, thereby weakening the frequencies. This indicates that the distortion correction manner can better distinguish the influence of field curvature on the results of the MTF evaluation.

Fig. 6

Corrected MTFs comparison with the theoretical MTF based on the geometric distortion correction manner.

5.

Conclusions

An LMTF method is successfully demonstrated for imaging evaluation of the MRE. This method establishes a mathematical model of geometric distortion and analyzes two sampling ROI image manners in MTF evaluation. Compared with the distorted factor manner, the geometric distortion correction manner relaxes the DF value to $2\times \alpha$, extends the sampling ROI image to twice that of the DF manner, reduces the average $R_{\mathrm{RMSE}}$ value to 6%, and improves average accuracy on MTF by 1.2%. The experimental results provide good agreement with the theoretical prediction. The SMTF method provides a feasible solution for the optical lenses with poorly corrected geometric distortion.