This is a list of reviewers who served the Journal of Electronic Imaging in 2021.

Training data for deep learning algorithms can have many redundancies, which should be resolved to achieve faster training speed and efficient storage usage. We proposed a random sample consensus (RANSAC)-based training data selection technique to reduce the training data size for deep learning-based image classification tasks. First, we formulate the data reduction problem as a least square problem and reformulate the equation as maximizing the accuracy of the total training set. Based on the reformulated equation, we applied an RANSAC algorithm to solve the optimization problem. We obtain superior or comparable accuracies to other data selection approaches, such as random, greedy k-means-based, and least square-based approaches. Notably, our algorithm was not degraded in small data selection, unlike other state-of-the-art algorithms.

In recent years, cylindrical millimeter-wave (MMW) holographic imaging technique has attracted a lot of attention for its special advantages, including high resolution, large aperture, and strong penetrability. A modified wavenumber-domain three-dimensional imaging algorithm applied for near-field cylindrical MMW holography is proposed. The core of the proposed algorithm is to generate an accurate and efficient phase compensation factor in wavenumber-domain. Compared with the existing algorithms, the proposed algorithm holds advantages in both imaging speed and imaging quality and is more suitable for security inspection application. Both the simulation results of point targets and the experimental results of the resolution test panel validate the effectiveness of the proposed algorithm.

Neuromorphic computing is becoming a popular approach for implementations of brain-inspired machine learning tasks. As a paradigm for both hardware and algorithm design, neuromorphic computing aims to emulate several aspects related to the structure and function of the biological nervous system to achieve artificial intelligence with efficiencies that are orders of magnitude better than those exhibited by general-purpose computing hardware. We provide a holistic treatment of spike-based neuromorphic computing (i.e., based on spiking neural networks), detailing biological motivation, key aspects of neuromorphic algorithms, and a survey of state-of-the-art neuromorphic hardware. In particular, we focus on these aspects within the context of brain-inspired vision applications. Our aim is to serve as a complement to several of the existing reviews on neuromorphic computing while also providing a unique perspective.

Robot intelligent inspection is widely used in the positioning of various pointer instruments in power, petroleum, chemical, and other industries. Aiming at the technical problems of poor adaptability, poor real-time performance, and low positioning accuracy of the pointer instrument positioning method in the existing substation intelligent inspection robot system, we propose a simple and effective pointer instrument positioning detection algorithm. The algorithm first extracts locally adaptive regression kernels (LARK) features of the input image, and the dimension of the LARK feature is reduced using the principal components analysis algorithm. Then, the template image is slid in the input image, the cosine similarity is used as an evaluation index, and the Fourier transform is used to accelerate the convolution operation in the cosine similarity calculation. Finally, the accelerated-KAZE algorithm is used to extract the feature points of the pointer-type instrument area image and the template image, and the statistical method of grid motion was used to eliminate the wrong matching points. The remaining matching points were processed by random sample consensus algorithm, and the homography matrix was obtained. The image registration was completed by the homography matrix, and the pointer-type instrument region positioning was realized. The experimental results show that the proposed method has good adaptability, strong real-time performance, and high accuracy of pointer-type instrument positioning.

The main task of image fusion is to extract the advantage information from the source images. Multiscale transform (MST) is commonly used in image fusion, but it is difficult to integrate detail information and structural information from source images. Therefore, it is important to integrate the information in each image patch group. Different from MST, edge preserving technology obtains more edge information than MST by extracting spatial information. Based on this superior feature, we propose an infrared and visible image fusion method via rolling guidance filter (RGF) and weight map. First, source images are decomposed into multiscale detail layers and structural base layers by RGF; second, to avoid the loss of local information, the modified Laplacian energy weight matrix is used as detail weight maps to merging the detail layers; then to reduce the loss of contours of edge objects, a structural weight maps are used to merging the base layers; finally, these two different weight maps are applied to reconstruct the fused image. Simulation results indicate that our fusion approach performs better in merging detail information and structural information than some classic MST fusion algorithms and the latest fusion methods.

We present a stereo matching approach referred to as HLocalExp-CM by exploiting the hierarchical local contextual information and a confidence map based on a new grid structure. The proposed approach preserves fine depth edges and extracts accurate disparities in weak texture, textureless, and repeated texture regions. The proposed approach adopts a two-stage optimization strategy. In the framework of first stage, a multiresolution cost aggregation is minimized to reduce the search space of the disparity plane of each pixel. The second stage iteratively optimizes the confidence map and a global energy function to progressively improve the disparity accuracy for each pixel. The confidence map is estimated through classifying the pixels into distinctive and ambiguous ones by computing the decreasing rate of the multiresolution cost aggregation and then performs a spatial propagation and plane refinement for the update of the disparity of each pixel, thereby successfully eliminating the ambiguity of nondistinctive pixels. The global energy function based on a pairwise Markov random field uses cross-scale cost aggregation for taking advantage of context information of objects in different scenarios on local grid regions, which is different from the deep learning technique uses convolution layers extracting the context information. The proposed approach is evaluated on Middlebury benchmark V3, and is ranked first based on “bad 2.0 all metric,” a widely used criterion for the evaluation of stereo images, while the eighth place on “bad 2.0 nonocc metric” (recorded on July 24, 2021).

The core challenge of few-shot learning is the serious overfitting problem on new tasks because of the scarcity of labeled data. Self-supervision learning can mine powerful supervision signals from the data itself to enhance the generalization performance of the model. Thus a rotation self-supervision module has been directly integrated to a few-shot learning network to alleviate the overfitting problem. However, due to the level difference or convergence speed difference in the loss function for each task, the overall model can be alternately dominated or biased by a certain task during the training stages, which has disadvantages for the main task performance. Therefore, we design a network architecture with auxiliary task learning speed equalization (LSENet) for few-shot learning. The overall model improves the generalization capability using an auxiliary task. In addition, we design a speed equalization module to constrain the decline rate of the two tasks to achieve balanced learning. Our method alleviates the overfitting problem of few-shot learning and greatly improves classification accuracy. Extensive experiments are conducted on benchmark datasets to demonstrate the effectiveness of our method.

In the field of public safety surveillance, suspects may disguise themselves by painting their faces, challenging the face recognition system. The well-developed ordinary, two-dimensional image face recognition methods rely mainly on the texture features of the face images, which leaves the traditional face recognition system relatively vulnerable to the camouflage attacks. We set our sights on three-dimensional (3D) alternatives. A 3D facial recognition method is proposed which extracts 3D features of each individual that are not related to face texture. Considering actual requirements, we applied binocular stereo matching to obtain 3D face point clouds. Hereafter, we included a spatial representation for classification and recognition of 3D face, based on which we then adopted a multicascade classifier. By reconstructing, extracting features, and identifying multiple 3D faces, comparison experiment results demonstrate that our proposed method discerned 3D faces correctly, defeated the face camouflage attacks effectively, and showed promising application prospects for public safety surveillance.

With the latest popularization of remote desktop software and online work, screen content images (SCIs) are increasingly being contacted by people. Many image quality assessment (IQA) methods based on deep learning have been proposed for SCIs. However, most IQA methods for SCIs pay less attention to regional characteristics. An asymmetric two-stream network based on region features is proposed for SCI objective quality assessment, in which features are derived from a channel separation feature extraction (CSFE) sub-network and a spatial analysis feature extraction (SAFE) sub-network. In the CSFE sub-network, three channel separation (CS) modules are utilized to strengthen the ability for capturing deepthwise features in the pictorial regions. In the SAFE sub-network, two spatial analysis (SA) module are applied to focus on the structural information in the spatial domain for the textual regions. Two asymmetric sub-networks can efficiently extract the corresponding region features by separately considering the pictorial and textual regions of SCIs. In addition, an adaptive global weighting strategy is introduced to estimate the quality for SCIs. The experimental results on the screen image quality assessment database show that the proposed model is superior to the other state-of-the-art SCI quality assessment methods.

Face anti-spoofing (FAS) is crucial for safe and reliable biometric systems. In recent years, deep neural networks have been proven to be very effective for FAS as compared with classical approaches. However, deep learning-based FAS methods are data-driven and use learning-based features only. It is a legitimate question to ask whether hand-crafted features can provide any complementary information to a deep learning-based FAS method. To answer this question, we propose a two-stream network that consists of a convolutional network and a local difference network. To be specific, we first build a texture extraction convolutional block to calculate the gradient magnitude at each pixel of an input image. Our experiments demonstrate that additional liveness cues can be captured by the proposed method. Second, we design an attention fusion module to combine the features obtained from the RGB domain and gradient magnitude domain, aiming for discriminative information mining and information redundancy elimination. Finally, we advocate a simple binary facial mask supervision strategy for further performance boost. The proposed network has only 2.79M parameters and the inference speed is up to 118 frames per second, which makes it very convenient for real-time FAS systems. The experimental results obtained on several well-known benchmarking datasets demonstrate the merits and superiority of the proposed method over the state-of-the-art approaches.

Image defogging has important application value in preprocessing technology and computer vision system. Dark channel prior (DCP) is simple and effective in many defogging algorithms, but the time-consuming sorting comparison and a large number of calculation refinement processes limit its real-time processing capabilities. For real-time applications, we proposed a hardware architecture for single-image defogging, which gives full play to hardware parallel processing capability and algorithmic parallelism. First, an average statistical approach is used to estimate atmospheric light. Then, the refined dark channel map is used for transmittance estimation to reduce the blocking effect. The transmittance is linearly corrected to prevent color distortion in outdoor scenes containing sky areas. Finally, a guided filter algorithm is introduced in the transmittance refinement, and its fast mean filter uses an adaptive window to process the image boundary. The hardware implementation of the proposed method uses field programmable gate array device is Zynq-7000. Experimental results show that our design obtains good performance with low-complex hardware implementation and shorter execution time. It only takes 7.43 ms to process a 1280  ×  720 image, and the frame rate can reach 135 fps at a clock rate of 125 MHz, which can be used as a real-time hardware accelerator for image processing.

Multiscale object detection is a challenging task due to the multiscale and multiclassification nature of different objects. Convolutional neural networks are commonly used to extract the features. However, continuous downsampling operations and spatial position quantization result in the loss of detailed information. Subsequently, the high-level feature maps cannot distinguish the features of adjacent objects, and in particular, small objects are ignored. To solve the above problems, we propose a model for multiscale object detection. First, we design a dual-bottleneck subconvolutional network to extract shallow features from the multiscale image pyramid. The subnetwork provides more spatial information for feature maps of the backbone network. Then, we introduce a new downsampling method to avoid losing a large amount of detailed information during the downsampling operations. Finally, we use a loss function to alleviate the foreground–background class imbalance. In the ablation study, we research the matching between the image pyramid module and the backbone network, and we compare different feature fusion modules. We evaluate our proposed model on two publicly available large-scale datasets and observe that our model achieves an 8.5% improvement compared with the classical feature pyramid network and exhibits acceptable detection performance for the occluded and diverse objects.

Analysis-synthesis dictionary pair learning has attracted much attention in the field of pattern classification. To reduce the negative effect of trivial information contained in raw training samples and improve the computation efficiency, most existing dictionary pair learning methods first learn a projection matrix to project raw training samples into a low-dimensional subspace and then use the dimensionality-reduced samples for dictionary pair learning. However, the separation of projection learning (PL) and dictionary pair learning may make the dimensionality-reduced training samples not fit well for dictionary pair learning. To address this issue, we proposed a joint PL and structured analysis-synthesis dictionary pair learning (PLSDPL) method for pattern classification. Specifically, PLSDPL integrates PL and dictionary pair learning into a unified framework, so the dimensionality-reduced training samples are more suitable for the structured analysis-synthesis dictionary pair learning, thus making the learned analysis-synthesis dictionary pair have powerful discrimination capability. In addition, PLSDPL also imposes a low-rank constraint on each synthesis sub-dictionary to reduce the negative effect of noise contained in training samples. Experimental results on several image datasets show that PLSDPL is effective for pattern classification tasks.

Image semantic segmentation is a fundamental problem in the field of computer vision. Although the existing semantic segmentation model based on fully convolutional neural network continuously optimizes the segmentation effect, the inherent spatial invariance of the network still leads to cause the loss of object edge details. Moreover, most models use the pixel-by-pixel loss to optimize the target, and the dependencies between pixels are ignored. When facing objects with smaller spatial structures in the image, the segmentation result is not satisfactory. Based on the theory of relative entropy and mutual information, we propose an overall objective loss function that integrates pixel similarity and image structure similarity. It can better pay attention to the structure and detail information of small objects in space by modeling the dependency relationship between pixels. We use the DeepLabv3+ network based on group normalization, with the improved ResNet50 as the backbone. After that, considering the particular advantages of superpixel segmentation for object edges, we propose a superpixel edge optimization algorithm, which combines pixel-level semantic features and superpixel-level regional information to obtain the semantic segmentation results after edge optimization. Experiments on PASCAL VOC 2012 and cityscapes datasets show that the proposed method improves the performance of semantic segmentation and shows better results in small target structures and object edge details.

Lightweight face detection algorithms that typically utilize convolutional neural network to find out all faces from the entire vision range. However, compared with more accurate and heavy algorithms, the performance of existing lightweight networks is still left far behind. Toward this end, we propose a lightweight and efficient single-stage face detector, named ACWFace, which explores the effects of attention, context module, and weighted feature fusion based on RetinaFace. First, efficient dual attention module is designed to further explore the potential of channel attention and spatial attention by introducing adaptive convolution kernel. Second, extended context module and shuffled context module are proposed to enlarge the receptive field and increase the information intersection between branches. Finally, weighted-fusion feature pyramid network is utilized to solve the features fusion of different scales equally by introducing the feature fusion module. Experiments on the easy, medium, and hard datasets of WIDER FACE validation partition show that our ACWFace outperforms RetinaFace average precision by 1.0%, 1.1%, and 1.8% while it achieves a great growth of 0.6%, 6.5%, and 3.0% on annotated faces in the wild, PASCAL face, and face detection data set and benchmark datasets, respectively.

Traditional image quality prediction methods require the pristine image as a reference, such as structural similarity. However, it is difficult to provide a haze-free image as a reference when predicting the quality of the dehazed image. Therefore, it is necessary to use a no-reference image quality assessment (NR-IQA) method. In addition, most NR-IQA methods are based on known distortion type, using a large number of subjective opinion scores and images with the same distortion to train the model. We developed an innovative NR-IQA specifically for dehazed images without such prior knowledge. Since most images will undergo color distortion and blur after dehazing, it is proposed to combine color and sharpness for evaluation. The quality of the image is evaluated on the HSI color space, where the H and S channels are utilized to evaluate color, and the I channel to sharpness. Experimental results show that the performance of the proposed metric is better than other existing evaluation methods for dehazed images.

Convolutional neural networks (CNNs) provide the sensing and detection community with a discriminative approach for classifying images. However, one of the largest limitations of deep CNN image classifiers is the need for extensive training datasets containing a variety of image representations. While current methods, such as generative adversarial network data augmentation, additions of noise, rotations, and translations, can allow CNNs to better associate new images and their feature representations to ones of a learned image class, many fail to provide new contexts of ground truth feature information. To expand the association of critical class features within CNN image training datasets, an image pairing and training dataset augmentation paradigm via a multi-sensor domain image data fusion algorithm is proposed. This algorithm uses a mutual information (MI) and merit-based feature selection subroutine to pair highly correlated cross-domain images from multiple sensor domain image datasets. It then re-augments the corresponding cross-domain image pairs into the opposite sensor domain’s feature set via a highest MI, cross sensor domain, and image concatenation function. This augmented image set then acts to retrain the CNN to recognize greater generalizations of image class features via cross domain, mixed representations. Experimental results indicated an increased ability of CNNs to generalize and discriminate between image classes during testing of class images from synthetic aperture radar vehicle, solar cell device reliability screening, and lung cancer detection image datasets.

Person re-identification (re-ID) is an important topic in computer vision. We study the one-example re-ID task, where each identity has only one labeled example along with many unlabeled examples. In practice, for the unlabeled data, it is difficult to differentiate each person because of many conditions, such as low resolutions, occlusions, and lighting. In previous works, fine-grained information has been proven to be useful for supervised re-ID. To solve choosing a reliable, easy sample for self-paced learning, we exploit fine-grained features to metric the distances between labeled data and unlabeled data, the combination strategy of global and overlapping-part distance selecting more positive data for model training. In addition, the attention mechanism has been introduced to suppress the interruption of background. The training data are split into three parts, i.e., labeled data, pseudolabeled data, and instance-labeled data. First, the model is initialized by one-shot data for each identity. Then, pseudolabels are estimated from the unlabeled data and updating the model iteratively. The self-paced progressive sampling method is adopted to increase the number of the selected pseudolabeled candidates step by step. Notably, with pretrained model, on Market-1501, the rank-1 accuracy of our method is 86.0% which exceeds most other methods, experiment on two image-based datasets demonstrate promising results under one example re-ID setting.

YOLOv4-tiny is a lightweight network designed for low-end devices. It proposes a global model scaling technology, which uses the same scaling method with few convolution filters in each stage of the network, resulting in a small receptive field and low accuracy. To solve this problem, we use different scaling techniques in shallow and deep network stages. For the shallow network stage, the Simple-StemBlock scaling module is proposed to simplify network based on factors such as FLOPs and network fragmentation. The module can effectively reduce computation and improve the diversity of features. For the deep network stage, we consider depth of network and hardware resource constraints, the Depth-CSPBlock scaling module is designed to expand receptive field while keeping the computation as low as possible and layered residual connection is built in the module to enrich semantic information. Besides, mish activation function is adopted in the backbone for higher accuracy. The experimental results show that the accuracy of the proposed method achieves 23.2% AP on MSCOCO test-dev and 66.73  %  AP₅₀ on VOC datasets, compared with YOLOv4-tiny, the AP and AP₅₀ increased by 3.3% and 3.7%, respectively. The speed of the proposed method on Jetson TX2 can reach 45.6 frames per second, which is 20% faster than YOLOv4-tiny.

Personnel detection in a coal mine is of great significance for monitoring the safety and real-time positioning of underground personnel. However, the uneven illumination, large dust, imaging noise, and other factors give the existing methods a high false detection rate for personnel detection. To overcome this problem, an improved YOLOv4 network for personnel detection based on thermal infrared images is proposed. First, the infrared personnel images are preprocessed by the method of super-resolution convolutional neural network to enhance their resolutions. Second, to improve the detection performance of the YOLOV4 network, the K-means++ clustering is used to optimize the anchor boxes estimation before the network training. Third, to further strengthen the generalization ability of the YOLOv4 and reduce the complexity of network training, the strategy of transfer learning is applied to train the YOLOv4 network. Experimental results show that on the infrared personnel image dataset underground coal mine, the improved YOLOv4 model achieves 96.25% and 48.2 fps in global AP and detection speed, respectively. Moreover, the improved YOLOv4 model has the advantages of strong robustness and generalization ability, which is very suitable for underground personnel detection, and provides a strong guarantee for the safety monitoring and management of coal mine personnel.

For complex images, interactive image segmentation can achieve better segmentation performance than other approaches. However, many classical interactive segmentation methods are sensitive to the locations and number of seed pixels. We introduce a seed extraction method to address this issue. First, we propose a robust seed measure to select and mark the seed pixels that cover the whole image. We use the center pixel of each superpixel as a seed. Then, each center pixel is labeled by our seed measure. Second, we propose a relabeling method for addressing isolated seeds. The extracted seeds can be used to locate the initial object area and can serve as the inputs of other interactive segmentation methods to reduce their dependence on user interaction. Extensive experiments on the Berkeley, GrabCut, and MSRA1000 datasets demonstrate the effectiveness of our seed extraction method.

Joint spatio-temporal feature learning is the key to video-based action recognition. Off-the-shelf techniques mostly apply two-stream networks, and they either simply fuse the classification scores or only integrate the high-level features. However, these methods cannot learn inter-modality relationship well. We propose a joint attentive (JA) adaptive feature fusion (AFF) network, a three-stream network that improves inter-modality fusion by exploring complementary and interactive information of two modalities, RGB and optical flow. Specifically, we design an AFF block to implement layer-wise fusion between both modality network channels and feature levels, where spatio-temporal feature representations with different modalities and various levels can be fused effectively. To capture three-dimensional interaction of spatio-temporal features, we devise a JA module by incorporating the inter-dependencies learned with the spatial-channel attention mechanism and combine multi-scale attention to refine the fine-grained features. Extensive experiments on three public action recognition benchmark datasets demonstrate that our method achieves competitive results.

We propose an innovative stereo matching algorithm using three constraints and collaborative optimization between pixels. First, to initialize a better disparity plane, the random points will be selected using matching energy stability, disparity stability, and left–right consistency as restrictions, instead of choosing the commonly used random sample consensus (RANSAC) method, which has certain randomness and affects the final optimization. Second, for the problem that the grid-based propagation method cannot effectively propagate the labels between adjacent similar pixels and its easiness to fall into local minimums, a cooperative competition mechanism between adjacent similar pixels is proposed, and the matching energy is minimized through collaborative optimization. Third, we iteratively optimize the parallax plane parameters and minimize the matching energy function, and thus improve the stereo matching accuracy. Finally, we test our method on Middlebury V3 test set, which achieves better performance against other state-of-the-art algorithms and ranks first in the matching error <1  pixel all metric, ranks second in the matching error <0.5  pixel all metric, and ranks third in the matching error <2  pixels all metric, and further the number of iterations is reduced and the processing speed was accelerated (recorded on May 3, 2021).

Micro-expression (ME), which reveals the genuine feelings and motives within human beings, attracts considerable attention in the field of automatic affective recognition. The main challenges for robust micro-expression recognition (MER) are from the short ME duration, low intensity of facial muscle movements, and insufficient samples. To meet these challenges, we propose an optical flow-based deep capsule adversarial domain adaptation network (DCADAN) for MER, which leverages a deep neural network stemming from these speculations. To alleviate the negative impact of the identity related features, optical flow preprocessing is applied to encode the subtle face motion information that is highly related to facial MEs. Then, a deep capsule network is developed to determine the part–whole relationships on optical flow features. To cope with the data deficiency and enhance the generalization capability via domain adaptation, an adversarial discriminator module that enriches the available samples from macro-expression data is integrated into the capsule network to train an expeditious end-to-end deep network. Finally, a simple and yet efficient attention module is embedded to the DCADAN to adaptively aggregate optical flow convolution maps into the primary capsule layers. We evaluate the performance of the entire network on the cross-database ME benchmark (3DB) using the leave-one-subject-out cross-validation. Unweighted F1-score (UF1) and unweighted average recall (UAR) are exploited as the evaluation metrics. The MER based on DCADAN achieves a UF1 score of 0.801 and a UAR score of 0.829 in comparison with a UF1 of 0.788 and a UAR of 0.782 for the updated approach. The comprehensive experimental results show that the incorporation of adversarial domain adaption into the capsule network is feasible and effective for representing discriminative features in ME and the proposed model outperforms state-of-the-art deep learning networks for MER.

Face inpainting is a challenging task in computer vision. Although deep learning-based methods that apply attention mechanism or utilize prior knowledge could reconstruct facial components, they may produce visual artifacts or lack detail texture. To solve mainly these two problems, we propose a multicolumn gated convolutional network (MGCN). MGCN is composed of three parallel branches with gated convolution to dynamically extract multispatial features, which could help to improve the global semantic coherence and achieve more effective performance in irregular mask. Specifically, for generating more plausible texture, we developed a diversified perceptual Markov random field to search correct feature patches in global rather than local images. Experiments on CelebA-HQ face and Flickr-Faces-HQ datasets demonstrate that MGCN achieves a more competitive performance than the state-of-the-art methods.

This study focuses on the over-fitting problem in the training process of the deep convolutional neural network model and the problem of poor robustness when the model is applied in an occlusion environment. We propose a unique data augmentation method, In-and-Out. First, the information variance is enhanced through dynamic local operation while maintaining the overall geometric structure of the training image; compared with the global data augmentation method, our method effectively alleviates the overfitting problem of model training and significantly improves the generalization ability of the model. Then through the dynamic information removal operation, the image is hidden according to the dynamic patch generated by multiple parameters. Compared with other information removal methods, our method can better simulate the real-world occlusion environment, thus improving the robustness of the model in various occlusion scenes. This method is simple and easy to implement and can be integrated with most CNN-based computer vision tasks. Our extensive experiments show that our method surpasses previous methods on the Canadian Institute for Advanced Research dataset for image classification, the PASCAL Visual Object Classes dataset for object detection, and the Cityscapes dataset for semantic segmentation. In addition, our robustness experiments show that our method has good robustness to occlusion in various scenes.

The transfer of an image can often lead to image noise, which degrades the quality of the image. There are various filter techniques that can be utilized to reduce or remove the noise in an image. There are two types of filtering methods that use the linear structure and the filtering method without considering the linear structure when denoising the image degraded by salt-and-pepper impulse noise. Filtering methods that consider the linear structure can effectively repair the linear structure in noisy images. However, previous methods that consider line structure have certain disadvantages. To overcome these disadvantages, we propose a new filtering method that considers the line structure for a monochrome image. In the proposed filtering method, two filtering windows with a line structure were considered. In the first filtering window, horizontal and vertical linear structures were considered while a linear structure in 12 directions was considered in the second one. Different linear filtering windows were selected for the proposed filtering method according to the different characteristics of the linear structure. When judging the linear structure, a judgment is made based on the information of two adjacent lines of the central line. The effectiveness of the proposed method was verified experimentally.

Deep learning-based image compressive sensing methods have received extensive attention in recent years due to their superior learning ability and fast processing speed. The majority of existing image compressive sensing neural networks use single-scale sampling, whereas multiscale sampling has demonstrated excellent performance compared to single-scale. We propose a multiscale deep network for compressive sensing image reconstruction that consists of a multiscale sampling network and a reconstruction network. First, we use convolution to mimic the linear decomposition of images, and the convolution is learned during the training process. Then a sampling network captures compressive measurements across multiple decomposed scales. The reconstruction network, which includes both the initial and enhanced reconstruction networks, learns an end-to-end mapping between the compressed sensing (CS) measurements and the recovered images of the network. Experimental results indicate that the proposed network framework outperforms the existing CS methods in terms of objective metrics, peak signal to noise ratio (PSNR), structural similarity index, and subjective visual quality. Specifically, at a 0.1 sampling rate, using 10 images for testing, and the average PSNR maximum (minimum) gain is 5.95 dB (0.25 dB).

The biometric system accepts a fingerprint of an individual to authenticate, authorize, or identify a person. However, some devices with a small sensor use a partial fingerprint as input. Recent research has shown that partial fingerprints are not unique compared to the full fingerprint, leading to the MasterPrint vulnerability. A MasterPrint is a partial fingerprint identifying at least 4% distinct subjects enrolled with the database. An adversary exploits such MasterPrints for executing financial frauds, mounting a presentation attack, or criminal activities. Our work proposes minutiae geometry-based partial fingerprint identification method targeted to alleviate the MasterPrint vulnerability. The experiments comprise partial datasets created from five full-fingerprint benchmark datasets and six existing approaches to compare the results of the proposed method. The observations show that the proposed approach achieved up to 97% identification accuracy while generating merely 0.1% MasterPrints.

With regard to inverse synthetic aperture radar imaging with limited bandwidth and sparse aperture, it is a challenge to traditional range-Doppler (RD) algorithm. We proposed an innovative two-dimensional (2D) joint sparse imaging algorithm, namely, 2D fast orthogonal matching pursuit (2D-FOMP) algorithm. In the proposed algorithm, one-dimensional OMP (1D-OMP) is extended to 2D-OMP in the complex domain from three aspects of atom recognition, projection update, and residual update. Then, the equivalence between 1D-OMP and 2D-OMP is analyzed theoretically. Meanwhile, two strategies that multi atom recognition and matrix recursive update are added in 2D-OMP to further improve the reconstruction speed of 2D-FOMP. Experimental results based on both simulated and measured data demonstrate that the proposed algorithm has good imaging performance under noisy and sparse conditions.

The total variation (TV) model preserves edges well but causes staircase effects and fails to protect textures. To avoid these limitations, an innovative hybrid regularization model that combines minmax-concave TV (and the shearlet sparsity is proposed for simultaneous image deblurring and image reconstruction. Although the proposed cost function is a non-convex L₁-regularized optimization problem, it can maintain the convexity of the cost function by giving the proper nonconvexity parameter to minimize it. Then, an alternating iterative scheme using variable splitting and the alternating direction method of multipliers is introduced to optimize the proposed model. The extensive experiments demonstrate the efficiency and viability of the proposed method in terms of both subjective vision and objective measures.

Object detection has achieved good progress in the last few years. However, there are many challenges in the field of remote sensing imagery. Objects in remote sensing images usually have arbitrary orientations and various scales. In addition, some objects are easily overwhelmed by a cluttered background. To take advantage of single-stage object detectors that have fast speed, many cascaded structures based on single-stage detectors have been proposed to improve detection performance. However, feature inconsistency in cascade structure results in poor detection performance. To address these problems, we propose an innovative model in terms of both model improvement and loss function refinement. This model consists of an attention module to highlight useful information in cluttered scenes, a multi-scale feature fusion module, and a cascade refinement module with anchor constrained convolution to address feature inconsistency. Furthermore, Intersection-over-Union (IoU) classification loss is proposed to enhance the correlation between classification and localization, and a scale-aware regression loss is proposed to improve the detection performance on objects with different scales. We conducted extensive experiments on both the DOTA dataset and the HRSC2016 dataset, and the experimental results show that our model has advantages compared with current state-of-the-art methods.

Powerful features, which contain more representative information, have become increasingly important in object detection. We exploit the attention mechanism and dilated convolution to strengthen the features used to construct the original feature pyramid network (FPN) and introduce a network that combines the dilated convolution and attention mechanism based on FPN (DAFPN). Specifically, motivated by the attention mechanism, a level-independent attention module (LIAM) is proposed to make high-level feature maps focus on semantic information and low-level feature maps concentrate on spatial information. Meanwhile, we present a pyramidal dilated convolution module (PDCM) that replaces standard convolution with dilated convolution. Instead of previous works that use the same dilation rate for all scales of feature maps, the PDCM applies dilation convolution with various dilation rates to enlarge the effective receptive field of each level’s feature maps suitably. Extensive experiments show that our DAFPN achieves extraordinary performance compared to the state-of-the-art FPN-based detectors on MS COCO benchmark.

To protect the security of image information in the process of storage or transmission, we propose an efficient image encryption method using chaos and game of life (GoL). First, a plaintext-related image scrambling method based on chaos and GoL is proposed to reduce the adjacent pixel correlation of the plain image. In this method, double scrambling is performed on the original image: the first scrambling matrix is constructed by logistic map and the rules of GoL; the second scrambling matrix is generated based on two-dimensional (2D) logistic map and magic square. The original image is scrambled by the above-mentioned scrambling matrices, which can efficiently reduce the strong adjacent pixel correlation, thus enhancing the image scrambling effect. Next, the 4D hyperchaotic system and GoL are employed to diffuse the scrambled image and get the cipher image. Additionally, the average pixel value of the original image is used to generate the initial values of the logistic map and 2D logistic map in scrambling process. Therefore, our encryption scheme is very sensitive to the plain image and can effectively resist the chosen-plaintext attack and known-plaintext attack. Finally, the experimental results and performance analyses show that the proposed encryption algorithm has the advantages of large key space and good scrambling effect, and it can effectively resist various typical attacks, so it can effectively protect the security of the image information.

Infrared and visible image fusion aims to fuse infrared targets and visible details in a composite scene. Recently, many fusion methods have been proposed, but most cannot balance characteristics of infrared and visible images well. We propose an infrared and visible image fusion method. First, to enhance the quality of the infrared and visible images to assist the subsequent prefusion and decomposition, a preprocessing method based on singular value decomposition combined with high dynamic range compression and guided filter (GF) is proposed. Then the preprocessing results are fused to produce the prefusion result by multiscale structural image decomposition-based fusion method. Next, the prefusion result is decomposed by the multilevel image decomposition method based on latent low-rank representation (MDLatLRR). Then the source images are decomposed by MDLatLRR. For the base layer, we combine the visual saliency map with GF to obtain the final fused base layer. For the detail layers, we propose a nonlinear function to highlight the infrared target and use weighted least square to optimize the details. We also adopt a structural similarity-based weight map via L₂-norm to improve the fusion performance. The experimental results demonstrate that the proposed method outperforms some state-of-the-art methods both qualitatively and quantitatively.

Image-to-image translation that refers to the synthesis task of transferring images from the source domain to the target domain has gained significant progress in recent times. Multimodal image-to-image translation aims to generate images with multiple styles of target domains. However, the existing multimodal image-to-image translation network architectures are incapable of accurately transferring the style of a specified image. Moreover, they require an additional deep encoder network to extract the image style code, which increases the network parameters. To address this problem, we propose Sunit, a multimodal unsupervised image-to-image translation with a shared encoder. Sunit shares an encoder network between the discriminator and style encoder. This method reduces the number of network parameters and uses the information from the discriminator to extract the style. Furthermore, we design a training strategy in which the style encoder solely uses the style reconstruction loss and does not follow the generator to train. In this manner, the target of the style encoder becomes clearer. Finally, extensive experimental validations are carried out on the AFHQ and Celeb-HQ datasets. The results demonstrate that our approach outperforms the state-of-the-art methods in the task of reference-guided image translation and transfers the style of the specified image more accurately.

We propose a robust gait recognition method based on the combination of global and local image entropy features. An improved feature extraction scheme is developed, in which binary walking silhouettes are characterized with global and local image entropy features. Gait dynamics underlying image entropy features are derived and fused. Additionally, pretrained deep neural networks are employed as the feature extractor on the raw fused image entropy features. The extracted gait dynamics and deep transfer learning features are finally fused and fed into a seven-layer fully connected network for the identification task. The proposed method can make use of global and local gait characteristics sufficiently, which is helpful for resisting walking conditions variation. Experiments on the CASIA-B database are conducted to demonstrate the efficiency of the proposed method.

As a new research hotspot in the realm of information security, reversible data hiding (RDH) technology has garnered a lot of attention and has been widely used in recent years. An RDH scheme with a high embedding capacity (EC) based on prediction-error expansion is proposed. Specifically, we first divide the host image into two regions of the same size. Then the local complexity value of each pixel is calculated. Furthermore, we employ five parameters, i.e., the local complexity threshold, the embedding bins on the left and right of the prediction error histogram (PEH) and their number to calculate the distortion of the image. The parameters corresponding to the minimal distortion are the optimal parameters. Finally, we embed secret data into the optimal embedding bins on PEH and obtain the best embedding performance. The experimental results reveal that our scheme has a higher EC and better performance than some other state-of-the-art methods.