“Flying spot” laser infrared thermography (FST) is a non destructive testing technique able to detect defects by scanning surfaces with a laser heat source. Defects, such as cracks on metallic parts, are revealed by the disturbance of heat propagation measured by an infrared camera. Deep learning approaches are now very efficient to automatically analyze and use contextual information from data and can be used for crack detection. However, in the literature only few works deal with the use of deep learning for the crack detection in FST. Indeed obtaining a large amount of data from FST examinations can be expensive and time-consuming. This work presents an open-access database for “flying spot” laser thermography, annotated for crack-type defect localization. It is followed by a benchmark of several state-of-the-art machine learning architectures. The benefits of this database for transfer to the detection and localization of cracks in coated metallic samples are highlighted, with very reduced amounts of data. Finally, this work gives a preliminary exploration on the use of individual thermal images, yielding to detection and localization performance comparable to the previously studied reconstructed thermal maps, which consists in the normalized mean of all the thermal images on a region of interest, after registration.

Visual inspection plays a pivotal role in numerous industrial production processes, and the pursuit of automation has surged with the rise of deep learning and convolutional neural networks (CNNs). Therein, the deployment of visual inspection CNNs on resource-constrained edge devices stands as a critical problem as these devices are the most affordable and well-suited for many industrial applications, e.g., production chains. Nonetheless, it faces challenges in meeting the computational demands of deep CNN models. Consequently, optimizing these models for efficient operation in such settings is imperative. Visual inspection tasks are often highly specialized, differing significantly from general computer vision tasks. As a result, state-of-the-art CNNs can be excessively large for achieving high accuracy on these specific datasets. To address this challenge, this paper introduces a novel approach utilizing neural architecture search (NAS) and hyperparameter optimization. We present the generic toolkit for NAS (GT-NAS), an open-source toolkit available for public use on GitLab ( https://gitlab.com/pmf5/open-source/generic-toolkit-for-neural-architecture-search). We showcase the results of applying our methodology to two established state-of-the-art CNN models designed for surface defect detection, a problem that encompasses binary classification and segmentation of images.Our approach yields significantly smaller models relative to baselines, but with accuracy in line with the current state-of-the-art results, demonstrating the potential for enhanced efficiency in industrial visual inspection systems. In one experimental setting (optimizing the Mixed Supervision model on the KolektorSDD2 dataset), GT-NAS produced an architecture that is 6.2 times faster than the baseline while sacrificing only 0.25% of its average precision for binary classification. In another batch of experiments (optimizing the TriNet model on the SensumSODF dataset), GT-NAS also achieved remarkable results. It found a TriNet architecture five times smaller than the baseline, at a small cost of a 0.25% drop in the ROC-AUC classification score on the capsule subset of the SensumSODF dataset. Furthermore, on the softgel subset of the same dataset, GT-NAS produced a model that was 2.7 times smaller than the baseline, yet 0.19% more precise.

Precision characterization is fundamental to achieve expected performance in semiconductors where Moore’s law pushes the boundaries to miniaturize components. To measure these attributes, deep learning models are used, which require manual annotation of several objects captured via electron microscopy. However, this annotation can be laborious and time-consuming. We propose a semi-automated method for annotating items in electron microscopy images, in an effort to be innovative, efficient, and reliable. Our approach involves identifying objects, enhancing boundaries with use of a unique loss function incorporating physical aspects from electron microscopy images. It greatly reduces the need for users to undertake the annotation model’s training process. It also minimizes post-inference processing by delivering a ready-to-use model. The constrained dynamic match loss (C-DML) incorporates dynamic matching with horizontal/vertical symmetry constraints to address the distinct challenges presented by manufactured objects acquired by microscopy imaging. Metrology metrics from the contour predictions obtained with C-DML obtain a mean relative error (MRE) of <10% and a correlation coefficient surpassing 90% when compared with ground truth corresponding to manual annotations. Our experimental results demonstrate the superior performance of C-DML over both classical DML and state-of-the-art deep annotation models. An extensive investigation demonstrates the effectiveness of our approach on heterogeneous datasets, including diverse objects of different materials and shapes, leading to state-of-the-art measurement results. Additionally, we show with the experiments that we can obtain better performances with better hyperparameters and data augmentation. Furthermore, this investigation presents a technique for annotating electron microscopy images efficiently and sheds light on the essential elements that dictate the approach’s overall efficacy.

Even though neural network methodologies have been established for a long time, only recently have they achieved exceptional efficacy in practical deployments, predominantly due to improvements in hardware computational capacity and the large amounts of available data for learning. Nonetheless, substantial challenges remain in utilizing deep learning in many domains, mainly because of the lack of large amounts of labeled data that are versatile enough for deep learning models to learn useful information. For instance, in mechanical assembly inspection, annotating data for each type of mechanical part to train a deep learning model can be very labor-intensive. Additionally, it is required to annotate data after each modification of mechanical part specification. Also, the system for inspection is typically not available until the first few samples are built to collect data. This paper proposes a solution for these challenges in case of the visual mechanical assembly inspection by processing point cloud data acquired via a three-dimensional (3D) scanner. To reduce the necessity for manually labeling large amounts of data, we employed synthetically generated data for both training and validation purposes, reserving the real sensor data exclusively for the testing phase. Our approach reduces the need for large amounts of labeled data by using synthetically generated point clouds from computer-aided design models for neural network training. Domain gap is a significant challenge for the usage of synthetically generated data. To reduce the domain gap, we used different preprocessing techniques, as well as a neural network architecture that focuses more on shared features that will not significantly change between synthetically generated data and real data from the 3D sensor.

Producing near net shape parts with complex geometries using wire-laser additive manufacturing (AM) often requires a mastered and optimized process. Differences between the constructed and nominal geometries of the manufactured entities demand an in-situ defects measurement to complete the production of the entire part successfully. A contactless measuring system is needed to evaluate geometrical deviations without requiring complex post-processing operations. To overcome this challenge and validate a measuring tool that serves the manufacturing purpose, a global stereocorrelation approach is used to measure defects in wire-laser additively manufactured parts. This method relies on the cameras’ self-calibration phase that uses the part substrate’s nominal model. Then, a modal basis is defined to model and evaluate the surface dimensional and shape defects. Hence, an analysis of the texture obtained in AM is conducted to assess whether or not it is sufficient for image correlation and defect measurement. Finally, natural and pattern textures are compared to highlight their influence on the measurement results.

Anomaly detection (AD) in an unsupervised manner has become the go-to approach in applications where data labeling proves problematic. However, these approaches are not completely unsupervised, since they rely on the weak knowledge of the dataset distribution into anomalous and anomaly-free subsets and typically require post-training threshold calibration in order to perform AD. Yet, they do not take advantage of available positive samples during training. In contrast, fully supervised approaches have proven to be more accurate and more efficient; however, they require a sufficient number of anomalous images to be labeled on a per-pixel level, which represents a labor-intensive task. In this article, we propose a hybrid approach that utilizes the best of both worlds. We use an unsupervised approach to build a model for generating pseudo labels, followed by a supervised approach to increase the robustness of AD. Moreover, we extend this approach with an active learning schema that results in learning with mixed supervision. We achieve several improvements, i.e., the utilization of available positive image samples, improved AD performance, and the retention of real-time performance. The proposed approach yields results that are comparable to the fully supervised approach, and at the very least, reduces the number of required labeled anomalous samples.

This paper aims to delineate a comprehensive method that integrates machine vision and deep learning for quality control within an industrial setting. The proposed innovative approach leverages a microservice architecture that ensures adaptability and flexibility to different scenarios while focusing on the employment of affordable, compact hardware, and it achieves exceptionally high accuracy in performing the quality control task and keeping a minimal computation time. Consequently, the developed system operates entirely on a portable smart camera, eliminating the need for additional sensors such as photocells and external computation, which simplifies the setup and commissioning phases and reduces the overall impact on the production line. By leveraging the integration of the embedded system with the machinery, this approach offers real-time monitoring and analysis capabilities, facilitating the swift detection of defects and deviations from desired standards. Moreover, the low-cost nature of the solution makes it accessible to a wider range of manufacturing enterprises, democratizing quality processes in Industry 5.0. The system was successfully implemented and is fully operational in a real industrial environment, and the experimental results obtained from this implementation are presented in this work.

Electrical cables consist of numerous wires, the three-dimensional (3D) shape of which significantly impacts the cables’ overall properties, such as bending stiffness. Although X-ray computed tomography (CT) provides a non-destructive method to assess these properties, accurately determining the 3D shape of individual wires from CT images is challenging due to the large number of wires, low image resolution, and indistinguishable appearance of the wires. Previous research lacked quantitative evaluation for wire tracking, and its overall accuracy heavily relied on the accuracy of wire detection. In this study, we present a long short-term memory-based approach for wire tracking that improves robustness against detection errors. The proposed method predicts wire positions in subsequent frames based on previous frames. We evaluate the performance of the proposed method using both actual annotated cables and artificially noised annotations. Our method exhibits greater tracking accuracy and robustness to detection errors compared with the previous method.

Throughout the steel production process, a variety of surface defects inevitably occur. These defects can impair the quality of steel products and reduce manufacturing efficiency. Therefore, it is crucial to study and categorize the multiple defects on the surface of steel strips. Vision transformer (ViT) is a unique neural network model based on a self-attention mechanism that is widely used in many different disciplines. Conventional ViT ignores the specifics of brain signaling and instead uses activation functions to simulate genuine neurons. One of the fundamental building blocks of a spiking neural network is leaky integration and fire (LIF), which has biodynamic characteristics akin to those of a genuine neuron. LIF neurons work in an event-driven manner such that higher performance can be achieved with less power. The goal of this work is to integrate ViT and LIF neurons to build and train an end-to-end hybrid network architecture, spiking vision transformer (S-ViT), for the classification of steel surface defects. The framework relies on the ViT architecture by replacing the activation functions used in ViT with LIF neurons, constructing a global spike feature fusion module spiking transformer encoder as well as a spiking-MLP classification head for implementing the classification functionality and using it as a basic building block of S-ViT. Based on the experimental results, our method has demonstrated outstanding classification performance across all metrics. The overall test accuracies of S-ViT are 99.41%, 99.65%, 99.54%, and 99.77% on NEU-CLSs, and 95.70%, 95.93%, 96.94%, and 97.19% on XSDD. S-ViT achieves superior classification performance compared to convolutional neural networks and recent findings. Its performance is also improved relative to the original ViT model. Furthermore, the robustness test results of S-ViT show that S-ViT still maintains reliable accuracy when recognizing images that contain Gaussian noise.

We introduce a method for efficient and expressive high-resolution image synthesis, harnessing the power of variational autoencoders (VAEs) and transformers with sparse attention (SA) mechanisms. By utilizing VAEs, we can establish a context-rich vocabulary of image constituents, thereby capturing intricate image features in a superior manner compared with traditional techniques. Subsequently, we employ SA mechanisms within our transformer model, improving computational efficiency while dealing with long sequences inherent to high-resolution images. Extending beyond traditional conditional synthesis, our model successfully integrates both nonspatial and spatial information while also incorporating temporal dynamics, enabling sequential image synthesis. Through rigorous experiments, we demonstrate our method’s effectiveness in semantically guided synthesis of megapixel images. Our findings substantiate this method as a significant contribution to the field of high-resolution image synthesis.

Image watermarking technology plays a key role in the protection of intellectual property rights. In addition to digital watermarking, optical watermarking has also been widely considered. A watermarking scheme based on Gerchberg–Saxton (GS) algorithm and integer wavelet transform (IWT) is proposed for image encryption. The scheme uses the unique phase reconstruction characteristics of GS algorithm, which makes it able to deal with a variety of complex local attacks in the process of protection and effectively restore the original image information. The obfuscation of position and numerical value information is realized by means of variable step Joseph space scrambling and pixel value bit processing. The carrier image is decomposed into subbands with different frequencies using IWT, all the information of the secret image is embedded bit by bit, which realizes the hiding of the image information. In addition, the SHA-256 function, the RSA algorithm, and the hyperchaotic system are combined to generate the cipher stream. The experimental results show that the algorithm has good imperceptibility and security, as well as strong robustness to cutting and salt and pepper noise attacks, and can restore the image quality well.

In the realm of low-resolution (LR) to high-resolution (HR) image reconstruction, denoising diffusion probabilistic models (DDPMs) are recognized for their superior perceptual quality over other generative models, attributed to their adept handling of various degradation factors in LR images, such as noise and blur. However, DDPMs predominantly focus on a single modality in the super-resolution (SR) image reconstruction from LR images, thus overlooking the rich potential information in multimodal data. This lack of integration and comprehensive processing of multimodal data can impede the full utilization of the complementary characteristics of different data types, limiting their effectiveness across a broad range of applications. Moreover, DDPMs require thousands of evaluations to reconstruct high-quality SR images, which significantly impacts their efficiency. In response to these challenges, a novel multimodal DDPM based on structured knowledge distillation (MKDDPM) is introduced. This approach features a multimodal-based DDPM that effectively leverages sparse prior information from another modality, integrated into the MKDDPM network architecture to optimize the solution space and detail features of the reconstructed image. Furthermore, a structured knowledge distillation method is proposed, leveraging a well-trained DDPM and iteratively learning a new DDPM, with each iteration requiring only half the original sampling steps. This method significantly reduces the number of model sampling steps without compromising on sampling quality. Experimental results demonstrate that MKDDPM, even with a substantially reduced number of diffusion steps, still achieves superior performance, providing a novel solution for single-image SR tasks.