This PDF file contains the front matter associated with SPIE Proceedings Volume 12857, including the Title Page, Copyright information, Table of Contents, and Conference Committee information.

Coded optical streaking combines spatial encoding and optical streaking in data acquisition. This synergy enables ultrahigh-speed imaging over a two-dimensional field of view in real time. This imaging paradigm can be embodied by using various optical components, achieving imaging speeds from thousands to trillions of frames per second.

The widespread presence and use of visual data highlight the fact that conventional frame-based electronic sensors may not be well-suited for specific situations. For instance, in many biomedical applications, there is a need to image dynamic specimens at high speeds, even though these objects occupy only a small fraction of the pixels within the entire field of view. Consequently, despite capturing them at a high frame rate, many resulting pixel values are uninformative and therefore discarded during subsequent computations. Neuromorphic imaging, which makes use of an event sensor that responds to changes in pixel intensities, is ideally suitable for detecting such fast-moving objects. In this work, we outline the principle of such detectors, demonstrate their use in a computational imaging setting, and discuss the computational algorithms to process such event data for a variety of applications.

Photo-magnetic imaging (PMI) is a novel diffuse optical imaging technique used to recover high resolution images of the optical absorption coefficient of bio-tissue. It uses near-infrared laser light to slightly warm up the tissue and measures the induced temperature using magnetic resonance thermometry (MRT). The measured temperature maps are then converted into absorption maps using a dedicated PMI image reconstruction algorithm. We present a convolutional neural network -based image reconstruction algorithm that improves the accuracy of the recovered absorption maps while reducing the recovery time. This approach directly delineates the boundaries of tumors on the MRT maps. These boundaries are then used to generate soft a priori information that will be employed to constrain the standard PMI image reconstruction algorithm. We evaluate the performance of the algorithm using a tissue-like phantom with an inclusion representing the presence of a potential tumor. The obtained results show that our new method can delineate the tumor region with an accuracy of ~96%.

We introduce a new, brain-inspired method for extracting high-resolution images of optically dynamic objects obscured by dense scattering media by combining a dynamic vision sensor (DVS) with neuromorphic computing techniques. Spike trains generated by the optical detection hardware form the principal currency for downstream neuromorphic processing, registering only photons emanating from the object while static background from the ambient media is suppressed. The information encoded in each pixel of the camera provides the spiking inputs into a deep spiking neural network via an autoencoder. Results from benchtop experiments suggest the neuromorphic approach as an efficient alternative to existing methods, with applications across medical imaging, optical communication, and microscopy.

Transmission matrix (TM) has applications ranging from imaging through scattering, communication, to multimode fiber imaging. TM retrieval for binary amplitude modulation recovers the TM from the intensity outputs of probing binary incident fields. However, the computational complexity limits the application for retrieval of large TM. We propose an efficient algorithm for TM retrieval with binary amplitude modulation. Our method designs the probing binary fields with convolution matrix and develops efficient retrieval algorithm based on fast Fourier transform (FFT). It improves the computational complexity by orders of magnitude. We verify the proposed algorithm with simulated data.

Multiphoton microscopy (MPM) provides high-resolution imaging of deep tissue structures while allowing for the visualization of non-labeled biological samples. However, photon generation efficiency of intrinsic biomarkers is low and this, coupled with inherent detection inaccuracies in the photoelectric sensors, leads to an introduction of noise in acquired images. Higher dwelling times can reduce noise but increase the likelihood of photobleaching. To combat this, deep learning methods are being increasingly employed to denoise MPM images, allowing for a more efficient and less invasive process. However, machine learning models can hallucinate information, which is unacceptable for critical scientific microscopy applications. Uncertainty quantification, which has been demonstrated for image-to-image regression tasks, can provide confidence bounds for machine learning-based image reconstruction tasks, adding confidence to predictions. In this work, we discuss incorporating uncertainty quantification into an optimized denoising model to guide adaptive multiphoton microscopy image acquisition. We demonstrate that our method is capable of maintaining fine features in the denoised image, while outperforming other denoising methods by adaptively selecting to reimage the most uncertain pixels in a human endometrium tissue sample.

The performance of an imaging system is limited by optical aberrations, which cause blurriness in the resulting image. Digital correction techniques, such as deconvolution, have limited ability to correct the blur, since some spatial frequencies in the scene are not measured adequately due to the aberrations (‘zeros’ of the system transfer function). Our work proves that the addition of a random mask to an imaging system removes its dependence on aberrations, resulting in no systematic zeros in the transfer function and consequently less sensitivity to noise during deconvolution. In simulation, we show that this strategy improves image quality over a range of aberration types, aberration strengths, and signal-to-noise ratios.

Raman spectroscopy is an inelastic scattering technique that measures the molecular vibrational states of a sample with little to no sample preparation. These vibrational states are molecule-specific, therefore different compounds can be identified through rapid analysis. Raman spectroscopy has been implemented in a variety of different research areas, for example, forensic analysis, pharmaceutical product design, material identification, disease diagnostics, etc. Although Raman spectroscopy has been demonstrated in various applications, it still has limitations with data processing due to its innate weak signals. Historically, chemometrics techniques have been widely used for Raman spectroscopy for preprocessing data such as feature extraction (or feature selection), and data modeling. These models are often generated by using analytical data from different sources, enhancing model discrimination and prediction abilities, but this is limited by how much data is provided. Our group has designed a portable A.I. Raman spectrometer using machine learning through training and deep learning. This spectrometer uses a miniature Raman spectrometer paired with a well plate reader for multiple and rapid sample measurement. As sample measurements are taken the system will implement machine learning software to preprocess and postprocess Raman spectral data. This will minimize the workload of complicated analysis on the condition that there exists sufficient training data. Implementing a well plate reader aids in data collection for the AI training by mimicking experiments for preprocess and adding Raman standards. Through machine learning as more data is provided the system will learn how to implement past data on new data sets, therefore minimizing the amount of time and analysis needed by human interaction.

The study aimed to combine an X-ray micro-computed tomography (μCT) with photoluminescence (PL) and convolutional neural network (CNN) assisted voxel classification and volume segmentation for tooth structural integrity assessment at the microcrack site and verify this approach with extracted human teeth. The samples were first examined using an X-ray μCT and segmented with CNN to identify enamel, dentin, and cracks. A new CNN image segmentation model was trained based on “Multiclass semantic segmentation using DeepLabV3+” example and was implemented with “TensorFlow”. Secondly, buccal and palatal teeth surfaces with microcracks and sound areas were selected to obtain fluorescence spectra illuminated with wavelengths of 325 nm (cw) and 266 nm (0.5 ns pulsed). The proposed approach – using X-ray μCT in combination with PL and CNN assisted segmentation – reveals the possibilities for tooth structural integrity assessment at the crack area with distinct precision and versatility and can be applied for all the teeth microstructure and surface mapping analysis.

On-chip lens-less ptychographic microscopy enables large field-of-view, high-resolution imaging of thin specimens by utilizing multiple intensity images acquired with multi-angle illumination and an iterative phase retrieval algorithm. Image reconstruction in lens-less ptychography, however, heavily relies on accurate forward model for image formation, and thus any discrepancies between forward model and experimental settings (e.g., mismatch in sample-to-detector distance or a slight tilt of object plane) would result in poor reconstruction image quality or inaccurate phase estimation. Here, we propose a deep learning-based autocalibration strategy for lens-less ptychographic microscopy, which does not require precise forward model and large training datasets. Our method is based on untrained neural network that incorporates parameterized physical forward model and system aberration.

Remote photoplethysmography (rPPG) is an effective technique for non-contact estimation of vital signs using video-based detection of exposed skin. It enables contactless healthcare services for clinical measurements and medical diagnosis. While existing rPPG methods primarily focus on enhancing robustness against interference factors like motion artifacts and illumination changes, limited attention has been given to the influence of the age factor on rPPG model performance. This study explicitly analyzed the impact of the age factor on the application of an rPPG method to the blood oxygen saturation (SpO₂) estimation. Two key observations were made. First, it was more challenging to estimate SpO₂ with rPPG for the elderly than for young individuals. Second, the performance of SpO₂ estimation for the elderly could be improved by dividing the training data into different age groups and exclusively training the model on data collected from the elderly. These observations highlighted that the age factor had a significant impact on rPPG methods, emphasizing the need for explicit consideration of the age factor in rPPG method design.

Automatic intraoral imaging-based assessment of oral conditions is important for clinical and consumer-level oral health monitoring. But there is a lack of publicly available intraoral datasets. To address this, we developed a StyleGAN2-based framework to generate synthetic 2D intraoral images. The StyleGAN2 network was trained on 3724 images with a Frechet Inception Distance 12.10. Dental professionals evaluated image quality and determined if images were real or synthetic. Approximately 83.75% of generated images were deemed real. We created a framework that utilizes pseudo-labeling to incorporate the StyleGAN2-synthesized 2D intraoral images into a tooth type classification model. Our experiments demonstrated that the StyleGAN2 synthesized images can effectively augment the training set and improve the performance of the tooth type classification model.

Due to the loss of phase information in images captured by intensity-only measurements, the numerical reconstruction of inline digital holographic imaging suffers from the undesirable twin-image artifact. This artifact presents as an out-of-focus conjugate at the virtual image plane and reduces the reconstruction quality. In this work, we propose a diffusion-based generative model that eliminates such defocus noise in single-shot inline digital holography. The diffusion-based generative model learns the implicit prior of the underlying data distribution by progressively injecting random noise in data and then generating high-quality samples by reversing this process. Although the diffusion model has been successful in various challenging tasks in computer vision, its potential in scientific imaging has not been fully explored yet, and one challenge is the inherent randomness in its reverse sampling process. To address this issue, we incorporate the underlying physics of image formation as a prior, which constrains the possible samples from the data distribution. Specifically, we include an extra gradient correction step in each reverse sampling process to introduce data consistency and generate better results. We demonstrate the feasibility of our approach using simulated and experimental holograms and compare our results with previous methods. Our model recovers detailed object information and significantly suppresses the twin-image noise. The proposed method is explainable, generalizable, and transferable to other samples from various distributions, making it a promising tool for digital holographic reconstruction.

Subconjunctival hemorrhage (SCH) is a prevalent ocular condition characterized by the accumulation of blood beneath the conjunctiva, resulting in a visible red patch on the eye’s surface. The appearance of SCH and the limited understanding of its progression can cause significant anxiety for patients. To address this issue and enhance ocular health management, we develop a deep learning-enabled monitoring approach that quantitatively tracks the SCH healing process through spectral reconstruction. Our approach comprises two key technical components. Firstly, automatic white balance algorithms are employed to estimate the light source’s color temperature and adjust image colors, minimizing the impact of varying lighting conditions. The SCH segmentation achieves an accuracy of 96.2 %, effectively avoiding interference from skin and eyelashes. Secondly, our monitoring approach evaluates SCH color changes, which are crucial for determining the stage of recovery. By learning a complex mapping function, the approach generates 31 hyperspectral bands (400–700 nm) by recovering the lost spectral information from a given RGB image. This process allows for a more detailed spectroscopic assessment of the affected area. The rich spectral signatures obtained from these hyperspectral images enable the classification of SCH into three distinct stages, reflecting the blood reabsorption process. This study is the first to apply deep learning-based spectral reconstruction to SCH determination, enabling evaluation of the recovery process through spectroscopic and quantitative analysis. This approach has the potential to improve daily patient care and promote better eye health control by offering more comprehensive monitoring of SCH progression.

The development of a deep learning framework specifically designed for the analysis of intraoral soft and hard tissue conditions is presented in this paper, with a focus on remote healthcare and intraoral diagnostic applications. The framework Faster R-CNN ResNet-50 FPN was trained on a dataset comprising 4,173 anonymized images of teeth obtained from buccal, lingual, and occlusal surfaces of 7 subjects. Ground truth annotations were generated through manual labeling, encompassing tooth number and tooth segmentation. The deep learning framework was built using platforms and APIs within Amazon Web Services (AWS), including SageMaker, S3, and EC2. It leveraged their GPU systems to train and deploy the models. The framework demonstrated high accuracy in tooth identification and segmentation, achieving an accuracy exceeding 60% for tooth numbering. Another framework for detecting teeth shades was trained using 25,519 RGB and 25,519 LAB values from VITA Classical shades. It used a basic neural network leading to 85 % validation accuracy. By leveraging the power of Faster R-CNN and the scalability of AWS, the framework provides a robust solution for real-time analysis of intraoral images, facilitating timely detection and monitoring of oral health issues. The initial results provide accurate identification of tooth numbering and valuable insights into tooth shades. The results achieved by the deep learning framework demonstrates its potential as a tool for analyzing intraoral soft and hard tissue parameters such as tooth staining. It presents an opportunity to enhance accuracy and efficiency in connected health and intraoral diagnostics applications, ultimately advancing the field of oral health assessment.

Tracking fluorescent objects through movies is a critical first step in quantifying electrical or molecular dynamics in cells. In many applications, it is necessary to track large numbers of fluorescent objects moving through tissue in a nonrigid manner. In this submission, we describe the use of a graph attention-based neural network to detect-and-link fluorescent neuronal nuclei in the brain of freely behaving worms (C. elegans). This approach allows us to reliably match on average 33% of the cells. When combined with a nonrigid registration algorithm that can leverage partial matches, this approach allows efficient tracking of all cells with substantially less manual intervention. Further work is needed to integrate this into previous registration pipelines.

Correlative imaging allows the investigation of biological samples under various aspects. We are working towards a correlated extreme ultraviolet (EUV), infrared (IR) and visible (VIS) fluorescence microscope in a single integrated setup. With ptychography, which is a lensless imaging technique, quantitative amplitude and phase information are obtained. EUV ptychography provides resolution in the nanometer scale and an excellent element contrast, but suffers from low throughput due to limited coherent photon flux. Using longer wavelengths such as UV/visible and Near-IR enables high-speed imaging with sub micrometer resolution. Further, the combination with fluorescence detection adds functional contrast with micrometer-scale resolution. Here, we demonstrate in a proof-of-concept experiment correlated ptychography-fluorescence microscopy in the visible range. By using the reconstructed beam from the ptychography measurement the fluorescence scanning map can be deconvolved, which significantly improves the resolution.

Machine learning is a powerful technique for analysing large-scale data and learning patterns, which provides high accuracy and shorter processing times. In this work, a machine learning algorithm (multinomial logistic regression) is used to predict the gene families from a human DNA sequence. 4380 sequences were converted into overlapping k-mers of length 6 to produce 232 414 k-mers. The data set was split into 80/20 train and test datasets, and the multinomial logistic regression model achieved a 93.9% accuracy in predicting 6 gene families within 0.24 seconds. The model was 94.8% precise, 93.9% sensitive, and had an f1-score of 94%. The developed model in this study offers an alternative approach for medical professionals to gain insights into genetic information carried within DNA segments. By leveraging machine learning techniques, accurate and efficient predictions of gene families can aid in understanding genetic characteristics and contribute to advancements in personalised medicine, diagnostics and genetic research.

Optical coherence tomography (OCT) has shown an affinity for imaging white matter using intrinsic signals. Combined with an automated vibratome and mosaic imaging, serial blockface histology (SBH) can yield whole-brain white matter images at high resolution. A current drawback of SBH is the lack of real-time information that complicates the localization of brain structures during imaging. To address this, imaged slices can be registered to a pre-existing 3D volume to provide more contextual information during the acquisition. 3D brain image registration is a process where a volume is aligned to a standard template to perform further analysis in a common reference frame. Without a full 3D volume, however, this slice-to-volume registration often proves difficult. The search space is large, and the limited information hampers existing algorithms. In this article, we present a neural network that predicts the 3D position of a 2D slice and aligns it to the corresponding slice in 3D template volume. The network uses a VGG16 backbone to extract features, followed by fully connected layers to predict the transformations. Six mouse brains at a resolution of 25μm, imaged using Serial OCT, have been used to train the network. The loss is calculated by taking the Euclidian distance between the predictions and the ground truth, which has been randomly sampled from the volume. Applications for this model are 2D to 3D slice registration, providing contextual information during serial OCT acquisitions such as the progress, or a parcelization of the current slice into its brain regions.

Significance: Transillumination imaging is crucial in diverse applications, from biometrics and medical diagnostics to material characterization. The challenge of scattering-induced blurring has fueled continuous research in the development of effective deblurring techniques. This study contributes to the field by introducing and evaluating a scattering deblurring model rooted in deep learning principles, addressing the intricacies of light-absorbing structures within turbid media.

Aim: The primary objective of this study is to evaluate the precision and robustness of the proposed scattering deblurring model in reconstructing three-dimensional complex structures within the scattering medium. Adopting a multidimensional approach, the study integrates deep learning principles to surpass the traditional deblurring method with point-spread function deconvolution, establishing a framework for achieving high-fidelity 3D reconstruction structure combined with the commonly filtered-back projection method.

Approach: Leveraging a diverse dataset of simulation images to expose the model to various scattering structures, the proposed scattering deblurring technique is based on the Fully Convolutional Network, Attention Res-Unet. The evaluation of the model’s performance incorporates critical metrics such as the intersection over union (IoU) and the contrast improvement ratio (CIR).

Results: The study demonstrates the effectiveness of the proposed scattering-deblurring model in mitigating scattering blur. Evaluation metrics, including a maximum IoU of 0.9737 and a CIR of 7, 166, underscore the superior performance of the proposed method compared to the deconvolution method in the entire angular range.

Conclusions: In conclusion, this study underscores the importance of adaptive imaging techniques that address the diverse and complex geometries encountered in biomedical optics. The proposed scattering-deblurring model, anchored in deep learning principles, presents promising results in enhancing the visualization of light-absorbing structures within turbid media.

Artificial intelligence algorithms are interesting solutions to automate the tedious manual counting of white blood cells by a specialist. Although interesting machine learning algorithms have been proposed for this task, there is a lack in the literature for high-accuracy methods (more than 99%) tested on larger datasets (more than 10 thousand images). This paper presents a segmentation and classification methodology, based on Random Forest and ResNet50, along with a comparison between ResNet models with different numbers of layers. The segmentation was tested in microscope-like images mounted using multiple single-cell images, widely available in online datasets, yielding 300×300 images to be classified by the residual network. For image classification, ResNet50 reached higher accuracies (99.3%, to the best of our knowledge, the higher accuracy for models with more than 1000 images), with the model size comparison pointing to model overfitting for larger models.

This paper aims to demonstrate a novel deep-learning network that addresses the prediction of breast tumors for diffuse optical imaging. Two learning schemes, signal encoder and image encoder, in the proposed network are designed for reconstructing optical-property images. The former processing method takes boundary data directly to deep networks, and predicts the optical-coefficient distribution, while the latter feeds images obtained by inverse image reconstruction with artifacts and sometimes hard-to-localized tumors. All 10,000 samples of synthesized homogeneous and heterogeneous phantoms were randomly selected for training, validation, and testing of performance. Twelve phantom samples were employed to justify its effectiveness in real applications.

In the study transfer learning was employed to adapt the previously developed deep networks, 1D_BNCNN and 2D_BNCNN, to handle elliptical phantoms in DOI. The network was fine-tuned using the newly acquired elliptical phantom dataset by leveraging the knowledge and pre-trained weights obtained from the circular phantom dataset. This approach can potentially enhance the realism and accuracy of DOT imaging, enabling more precise characterization of biological tissues and structures.

Lensless imaging can drastically relax traditional camera constraints by replacing lenses with optical masks, enabling lighter, cheaper, and thinner systems. However, unlike lenses, there is a lack of clear criteria for optical mask design. Most approaches are heuristic: either selecting a random mask or designing one with desired spectral and/or directional filter properties. Recent work jointly optimizes a phase mask and a task-specific neural network, but in simulation. We propose and demonstrate hardware-in-the-loop (HITL) training for jointly optimizing the mask and reconstruction parameters of a lensless imaging system, using the physical system itself in forward passes and a simulated model for determining updates during backpropagation. As the physical system uses a programmable mask, system updates can be done during training. Results show significant improvements in image quality metrics (2.14 dB in PSNR, 21.4% relative improvement in a perceptual metric) by jointly learning mask and reconstruction parameters. A low-cost prototype (less than 100 USD) is used, with open-source training and measurement code available on GitHub: https://github.com/LCAV/LenslessPiCam

Undesired fringes can appear in interferometric phase measurement, leading to a degradation of contrast and resolution in the retrieved quantitative phase of measured cells and materials. Strong fringes can also introduce significant phase discontinuities, thereby increasing the complexity and time required for phase unwrapping. These fringes often originate from factors such as light reflection between material surfaces of optical devices. Addressing this issue typically necessitates more intricate optical designs or advanced devices, but this comes at the cost of extended design periods and increased expenses. Achieving complete elimination of these fringes may be not always feasible. In this context, we propose an efficient method to mitigate their influences and enhance object contrast and resolution. This method involves modeling the fringes and appropriately determining their frequency support. Primarily based on Fourier filtering, this approach has been successfully demonstrated using real-world interferometric data.

As a traditional Chinese medicine practice, acupuncture has long been shown to benefit pain and stress relief (especially for elderly people with chronical cases). Therefore, acupuncture is an important and effective alternative medical therapy for disabled elderly population living in areas of low healthcare coverage, which has become a more and more serious social problem as the Chinese population ages rapidly. However, training of acupuncturists is quite expensive and time consuming. With the arrival of the era of AI, how to automate the process of acupuncture treatment and minimalize the involvement of human labor has emerged as a great challenge and opportunity. This research studies a prerequisite of automatic acupuncture treatment: patient in-position detection during the acupuncture treatment process. We propose a fast and accurate one-stage anchor-free DNN model for patient in-position detection. Our model is an improvement of the basis model, YOLO X. The proposed framework consists of a backbone of CSP-DarkNet, a neck of feature pyramid network and a Decoupled Head. As for loss function, we combine the CIoU and the alpha-IoU losses to inherit both their advantages. A simplified version of the advanced label assignment technique of OTA, as well as data augmentation strategies of Mosaic and MixUp are utilized to improve the algorithm performance. Results on a self-collected dataset of acupuncture treatment (named as ATPD Dataset) show that our algorithm significantly outperform other state-of-the-art methods in the literature that are either multiple-staged or single-staged.