Three-dimensional (3D) imaging with structured light is crucial in diverse scenarios, ranging from intelligent manufacturing and medicine to entertainment. However, current structured light methods rely on projector–camera synchronization, limiting the use of affordable imaging devices and their consumer applications. In this work, we introduce an asynchronous structured light imaging approach based on generative deep neural networks to relax the synchronization constraint, accomplishing the challenges of fringe pattern aliasing, without relying on any a priori constraint of the projection system. To overcome this need, we propose a generative deep neural network with U-Net-like encoder–decoder architecture to learn the underlying fringe features directly by exploring the intrinsic prior principles in the fringe pattern aliasing. We train within an adversarial learning framework and supervise the network training via a statistics-informed loss function. We demonstrate that by evaluating the performance on fields of intensity, phase, and 3D reconstruction. It is shown that the trained network can separate aliased fringe patterns for producing comparable results with the synchronous one: the absolute error is no greater than 8 μm, and the standard deviation does not exceed 3 μm. Evaluation results on multiple objects and pattern types show it could be generalized for any asynchronous structured light scene.
When using traditional phase-shift profilometry for 3D measurement, it is necessary to keep the measured object static during the shooting process. When the measured object is moving, errors will occur if the projection and capture of the fringe image is not fast enough. This paper proposes a new method to reconstruct the moving object by double sampling. A trigger control device is applied to the camera and projector, which ensures that after each projection, two consecutive images are captured before the next projection. Then, the phase information is retrieved by analyzing the relationship between the motion and fringe patterns. Finally, the moving object is retrieved successfully. The proposed method increased the frame rate of the moving object reconstruction.
Phase shifting profilometry (PSP) shifts the phase by projecting multiple fringe patterns with different initial phase values. Errors will be introduced when the dynamic object moves among the multiple fringe patterns. This paper presents a new 3D reconstruction method of moving object based on PSP by projecting single fringe pattern. The phase shift is caused by the object movement. Multiple images of the object with movement are captured. The object movement is tracked first; then, the phase variation caused by the movement in the single projected fringe pattern is analyzed and the reconstruction model describing the fringe pattern with movement is given; at last, the phase value is retrieved by utilizing the phase variation caused by the movement. The effectiveness of the proposed method is verified by the simulations and the experiment.
When implementing the phase shifting profilometry to reconstruct the object, phase unwrapping is a critical step when the object with height jump is reconstructed. In this paper, a new method encoding the phase shift value is proposed to unwrap the phase map by two-frequency unwrapping method. Only four fringe patterns are required and the phase value of the low frequency is encoded into the phase shift value of the projected fringe patterns. The effectiveness of the proposed method is verified by the simulations.
Measuring surface deformation of objects with natural patterns using digital image correlation (DIC) is difficult due to the challenges of the pattern quality and discriminative pattern matching. Existing studies in DIC predominantly focus on the artificial speckle patterns while seldom paying attention to the inevitable natural texture patterns. We propose a recursive-iterative method based on salient features to measure the deformation of objects with natural patterns. The method is proposed to select salient features according to the local intensity gradient and then to compute their displacements by incorporating the inverse compositional Gauss–Newton (IC-GN) algorithm into the classic image pyramidal computation. Compared with the existing IC-GN-based DIC technology, the use of discriminative subsets allows avoidance of displacement computation at pixels with poor spatial gradient distribution. Furthermore, the recursive computation based on the image pyramid can estimate the displacements of the features without the need for initial value estimation. This method remains effective even for large displacement measurements. The results of simulation and experiment prove the method’s feasibility, demonstrating that the method is effective in deformation measurement based on natural texture patterns.
KEYWORDS: Fringe analysis, Speckle pattern, Speckle, Principal component analysis, 3D metrology, Composites, Error analysis, Shape analysis, Superposition, Signal to noise ratio
Phase unwrapping is one of the key steps for fringe projection profilometry (FPP)-based 3D shape measurements. Conventional spatial phase unwrapping schemes are sensitive to noise and discontinuities, which may suffer from low accuracies. Temporal phase unwrapping is able to improve the reliability but often requires the acquisition of additional patterns, increasing the measurement time or hardware costs. This paper introduces a novel phase unwrapping scheme that utilizes composite patterns consisting of the superposition of standard sinusoidal patterns and randomly generated speckles. The low-rankness of the deformed sinusoidal patterns is studied. This is exploited together with the sparse nature of the speckle patterns and a robust principal component analysis (RPCA) framework is then deployed to separate the deformed fringe and speckle patterns. The cleaned fringe patterns are used for generating the wrapped phase maps using the standard procedures of phase shift profilometry (PSP) or Fourier Transform profilometry (FTP). Phase unwrapping is then achieved by matching the deformed speckle patterns that encode the phase order information. In order to correct the impulsive fringe order errors, a recently proposed postprocessing step is integrated into the proposed scheme to refine the phase unwrapping results. The analysis and simulation results demonstrate that the proposed scheme can improve the accuracy of FPP-based 3D shape measurements by effectively separating the fringe and speckle patterns.
Fringe projection profilometry (FPP) has attracted considerable interests for addressing the challenge of measuring three-dimension (3D) shapes of moving objects. Compared with phase shift profilometry (PSP) which requires the capture of multiple fringe patterns and is thus only suitable for static objects, Fourier transform profilometry (FTP) is less sensitive to motion-induced errors. However, FTP is prone to the influence of background lights and variations of the surface reflectivity, which may result in less accurate measurements. There are studies aimed to reduce the measurement errors with FTP using more sophisticated processing of the fringe patterns. However, existing works focus on schemes based on single images and the correlation of the dynamic 3D shapes is largely unexplored. In this work, we present a new method that refines FTP-based dynamic shape measurements. Assuming 3D rigid movements of the targets, we propose to utilize knowledge of the motion parameters and combine the multiple height maps obtained from several FTP measurements after compensating the motion effect. Approaches for automatically combining the height information are studied. It is observed that the measurement accuracy can be improved using the proposed method and the influence due to ambient lights and reflectivity variations can be suppressed. Computer simulations are performed to verify the effectiveness of the proposed method. The proposed method can also be integrated into other FPP systems to improve the performance for dynamic object measurements.
Phase unwrapping is an important step for the phase shifting profilometry. The dual-frequency phase unwrapping method can unwrap the object with discontinues when the object is static by employing more fringe patterns. However, errors will occur when moving object is reconstructed. In this paper, a new phase unwrapping method with dual-frequency phase unwrapping method for the moving object measurement is proposed. The fringe pattern with low fringe pattern and high frequency are projected onto the moving object surface. Then, the phase values are retrieved for the two frequencies respectively. The relationship between the movement and phase value is analyzed and the phase variations caused by the movement is compensated. At last, the phase value is unwrapped by the traditional dual-frequency phase unwrapping method. The effectiveness of the proposed method is verified by simulations.
Phase retrieve is an important step for phase shifting profilometry (PSP). The existing phase retrieve methods can obtain the phase value successfully for static object. However, as multiple fringe patterns are required in PSP, when the object has movement, errors will be introduced. A new phase retrieve method for the object with 2D movement is proposed in this paper. The 2D movement is divided into translation movement and rotation movement. Then their influence on the phase value is analyzed and a new reconstruction model including the movement information is given. At last, the phase value is retrieved based on the new reconstruction model. The proposed method can eliminate the errors caused by 2D movement of object. The effectiveness of the proposed method is verified by simulations.
In fringe projection profilometry, three-dimensional reconstruction result with higher spatial resolution could provide more detailed description of the measured object. To increase the spatial resolution of three-dimensional reconstruction result, this paper proposes a method to improve the resolution of the absolute phase map recovered in fringe projection profilometry with the digital images of different resolutions. In this method, the same fringe patterns are sampled with the images of different resolutions, the absolute phase maps of different resolutions are obtained respectively. Since the sampling points of the digital images under different resolutions are not coincident, additional ground truth depth information of the object surface is obtained. To emerge an absolute phase map with higher resolution, an algorithm is developed to fuse the absolute maps in different spatial resolutions together. The proposed method can be used to obtain a three-dimensional reconstruction result with higher spatial resolution for various applications. The effectiveness of our proposed method is validated by simulation results.
This work proposes a two-step phase-shifting algorithm as an improvement of fringe projection profilometry. Considering the working process of fringe projection, the captured fringe image is formulated with two variables, i.e. surface reflectivity and phase value. And a phase shift of 3π/2 is introduced to get the two-step phase-shifting. After appropriate variable substitution, expressions of two fringe images can be transformed into two equations corresponding to a line and a circle respectively. With this circle-line model, the characteristic of solution and the phase error due to non-zero ambient light are analyzed. Then the approach of error compensation is proposed based on estimation of the real fringe contrast and non-linear least square optimization. The validity of the proposed approach is demonstrated with both simulations and experiments.
Fringe pattern profilometry using triangular patterns and intensity ratios is a robust and computationally efficient method in three-dimensional shape measurement technique. However, similar to other multiple-shot techniques, the object must be kept static during the process of measurement, which is a challenging requirement for the case of fast-moving objects. Errors will be introduced if the traditional multiple-shot techniques are used directly in the measurement of a moving object. A new method is proposed to address this issue. First, the movement of the object is measured in real time and described by the rotation matrix and translation vector. Then, the expressions are derived for the fringe patterns under the influence of the two-dimensional movement of the object, based on which the normalized fringe patterns from the object without movement are estimated. Finally, the object is reconstructed using the existing intensity ratio algorithm incorporating the fringe patterns estimated, leading to improved measurement accuracy. The performance of the proposed method is verified by experiments.
Phase shifting profilometry (PSP) technique is widely used as a 3-D shape measurement technique due to its robustness
and accuracy. However, PSP requires multiple fringe pattern images to be projected onto an object and a reference plane
to calculate the phase value, and also the object must maintain motionless when the measurement is taken. If the object
moves during the measurement, significant errors will be introduced when calculating the phase value. This paper
analyses the relationship between the object movement and the phase value, and proposes a method for compensating the
errors caused by two-dimensional movement of object. This method can eliminate the errors caused by two-dimensional
movement of object and reconstruct the object shape successfully. The effectiveness of the proposed method is verified
by simulations.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.