KEYWORDS: Multiphoton microscopy, Image quality, In vivo imaging, Data modeling, Brain, Neuroimaging, Stereoscopy, Signal to noise ratio, Mathematical modeling, Neural networks
Multiphoton microscopy is a very important and powerful technique in the application of in-vivo drosophila brain imaging. However, the photon budget, which is compromising between imaging speed, spatial resolution and accumulating time, remains a huge challenge in biological compatibility. Rapid dual-resonant volumetric multiphoton microscopy combines a tunable acoustic gradient (TAG) lens and a resonant mirror, which can achieve up to 8 kHz frame rate and hundreds of hertz volumes per second temporal resolution. Adaptively sampling each laser pulse by an embedded field programmable gate array (FPGA) with up to 80 MHz pixel rate enables efficient but restricted signal accumulation. This study has developed a generative neural network to restore images from degradation of low signal-to-noise ratio (SNR), missing pixels and pattern residual. A series of training data by adding noise and Lissajous scanning path into the known fluorescent bead images were used to pretrain the model for the rapid dual resonant volumetric multiphoton microscopy system. Experimental results verify that the axial distortion and the image resolution of noisy fluorescent bead images can be effectively restored with the deep-restoration neural network. The mushroom body (MB) of drosophila brain which contains thousands of Kenyon cells in a 200 × 200 × 100 µm3 volume is utilized to demonstrate the strategy. The deep-restoration rapid dual resonant volumetric multiphoton microscopy image not only maintains 256 × 256 × 128 voxels and ~30 volumes per second, but also significantly improves image quality which is compatible to the ground truth. However, in-vivo imaging is time-varying. Base on in-vitro image datasets, in-vivo images via transfer learning is utilized to enables fast and improved image quality efficiently. Despite of one pulse per pixel, in-vivo drosophila brain imaging with deep-restoration not only keeps the advantage of temporal resolution, but also obtains well image quality.
The imaging speed of Temporal focusing multiphoton excitation microscopy (TFMPEM) is up to hundreds frames rate. However, the plane illumination manner suffers from the sever scattering of biotissue and signal crosstalk that blurs the image. And the deeper the worse. Nevertheless, the high acquisition rate decreases the effective excited fluorescent, which reduces the signal-to-noise ratio (SNR) of the image. In order to solve the scattering and low SNR issues, the deep learning method is proposed to restore the TFMPEM image. In this work, we construct a powerful neuron network which called multi-stage 3D U-Net. Different from the cascade method, it becomes more connection between each U-Net. The previous stage information can share with the next stage, instead of seeing as independent. Thus, we try to restore the TFMPEM via this network with Point scanning multiphoton excitation microscopy (PSMPEM) image as the ground truth. But before that way, our two systems are not sharing the same optical path architecture, it needs to do the registration first. For cross modality registration, we utilize Voxelmorph which is also a 3D U-Net architecture. And it can do the not only global but also local deformation, is flexible than classical algorithm. Hence, we do the registration and restoration via all deep learning method. Therefore, the peak signal-to-noise ratio (PSNR) of the image can be improved around 20 to 30 dB and, and structural similarity (SSIM) is close to 0.9
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.