High-accuracy and high-speed 3D sensing technology plays an essential role in VR eye tracking as it can build a bridge to connect the user with virtual worlds. In VR eye tracking, fringe projection profilometry (FPP) can avoid dependence on scene textures and provide accurate results in near-eye scenarios; however, phase-shifting based FPP faces challenges like motion artifacts and may not meet the low-latency requirements of eye tracking tasks. On the other hand, Fourier transform profilometry can achieve single-shot 3D sensing, but the system is highly impacted by the texture variations on the eye. As a solution to the challenges above, researchers have explored deep learning-based single-shot fringe projection 3D sensing techniques. However, building a training dataset is expensive, and without abundant data the model is difficult to make generalized. In this paper, we built a virtual fringe projection system along with photorealistic face and eye models to synthesize large amounts of training data. Therefore, we can reduce the cost and enhance the generalization ability of the convolutional neural network (CNN). The training data synthesizer utilizes physically based rendering (PBR) and achieves high photorealism. We demonstrate that PBR can simulate the complex double refraction of structured light due to corneas. To train the CNN, we adopted the idea of transfer learning, where the CNN is first trained by PBR-generated data, then trained with the real data. We tested the CNN on real data, and the predicted results demonstrate that the synthesized data enhances the performance of the model and achieves around 3.722 degree gaze accuracy and 0.5363 mm pupil position error on an unfamiliar participant.
This paper proposes a single-camera line-scan hyperspectral four-dimensional imaging technique, which can simultaneously retrieve accurate 3D geometrical information and high-resolution hyperspectral information. The system contains a camera attached to a line spectrograph, a video projector, and a linear guideway. Then the mathematical model for line-scan fringe projection profilometry as well as the 3D reconstruction and spectral calibration methods are investigated. The system can simultaneously achieve high spectral resolution (2.8 nm) and geometric measurement accuracy (0.0895 mm). Test cases will demonstrate that the system can obtain rich spectral and surface topographical information and has potential application in the food industry.
High-speed three-dimensional (3D) fringe projection profilometry (FPP) is widely used in many fields. Recently, researchers have successfully tested the feasibility of performing fringe analysis using deep convolutional networks (CNN). However, the existing methods require tremendous real-world scanning trials for model training, which is not trivial. In this work, we propose a framework to establish the digital twin of a real-world system in a virtual environment and a process to automatically generate 3D training data. Experiments are conducted to demonstrated that a physical system can adopt the CNN trained in the virtual environment to perform accurate real-world 3D shape measurements.
This paper introduces a novel uniaxial fringe projection profilometry (FPP) called active shape from projection defocus profilometry (ASPDP), which utilizes the sharpness analysis of binary fringe patterns to quantify the defocus level. Compared with previous uniaxial FPP methods, our work first utilizes a pinhole defocus model to account for three-dimensional reconstruction. Since defocused fringe pattern can be modeled as the original pattern convoluted with a point spread function (PSF), pixel-wise defocus level can be quantified with this PSFs kernel using temporal Fourier analysis. In this research, calibration is achieved by using a mechanical translation device, and determined by rational polynomial fitting to establish defocus-depth relationship. The experiment demonstrates that this method can provide an accurate reconstructed 3D geometry without shadows.
Three dimensional (3D) topology data obtained from different optical metrology techniques tend to produce local disagreements which may yield incorrect judgement from inspectors especially under scenarios of precision metrology. This research explores statistical methods to provide a functional scoring for similarities. The investigation is conducted using two statistical methods (Pearsons correlation coefficient and image distance), two optical techniques (structured light and focus variation microscopy) and two application scenarios (metal additive printing and ballistic forensic examination). Experimental results show the promise of using statistical tools to assist binary decisions for matching/non-matching even if 3D topology data are obtained from different optical techniques.
High-speed and high-accuracy three-dimensional (3D) measurement plays an important role in numerous areas. The recent proposed binary defocusing techniques have enabled the speed breakthrough by utilizing 1-bit binary fringe patterns with the advanced digital light processing (DLP) projection platform. To enhance the phase quality and measurement accuracy, extensive research has also been conducted to modulate and optimize the binary patterns spatially or temporally. However, it remains challenging for such techniques to measure objects with high dynamic range (HDR) of surface reflectivity. Therefore, to overcome this problem, this paper proposes a novel HDR 3D measurement method based on spectral modulation and multispectral imaging. By modulating the illumination light and acquiring the fringe patterns with a multispectral camera, high-contrast HDR fringe imaging and 3D measurement can be achieved. Experiments were carried out to demonstrate the effectiveness of the proposed strategy.
This paper introduces a novel real-time high dynamic range 3D scanning method with RGB camera, which utilizes the cameras varying color sensitivity and the projector’s dark time to alleviate saturation-induced measurement error. The varying color responses in R, G, and B channels creates three different intensity levels, and an additional capture at the projectors bright to dark transition state doubles the total intensity levels to six. Finally, saturation-induced errors can be alleviated by choosing the unsaturated pixel of best quality among the images with six intensity levels. Experimental results will be presented to demonstrate the success of such method.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.