Super resolution (SR) is to produce a higher resolution image from one or a sequence of low resolution images of a scene. It is essential in medical image analysis as a zooming of a specific area of interest is often required. This paper presents a new multi-frame super resolution (SR) method that is robust to both global and local motion. One of major challenges in multi-frame SR is concurrent global and local motion emergent in the sequence of low resolution images. It poses difficulties in aligning the low resolution images, resulting in artifacts or blurred pixels in the computed high resolution image. We solve the problem via a series of new methods. We first align the upscaled images from bicubic interpolation, and analyze the pixel distribution for the presence of local motion. If local motion is identified, we conduct the local image registration using dense SIFT features. Based on the local registration of images, we analyze pixel locations whose cross-frame variation is high and adaptively select subset of frame pixels in those locations. The adaptive selection of frame pixels is based on a clustering analysis of luminance values of pixels aligned at the same position, such that noise and motion biases are excluded. At the end, a median filter is applied for the selected pixels at each pixel location for super resolution image. We conduct experiments for multi-frame SR, where the proposed method delivers favorable results, especially better than state-of-the-art in dealing with concurrent local and global motions across frames.
In this paper, we develop a server-client quantization scheme to reduce bit resolution of deep learning architecture, i.e., Convolutional Neural Networks, for image recognition tasks. Low bit resolution is an important factor in bringing the deep learning neural network into hardware implementation, which directly determines the cost and power consumption. We aim to reduce the bit resolution of the network without sacrificing its performance. To this end, we design a new quantization algorithm called supervised iterative quantization to reduce the bit resolution of learned network weights. In the training stage, the supervised iterative quantization is conducted via two steps on server – apply k-means based adaptive quantization on learned network weights and retrain the network based on quantized weights. These two steps are alternated until the convergence criterion is met. In this testing stage, the network configuration and low-bit weights are loaded to the client hardware device to recognize coming input in real time, where optimized but expensive quantization becomes infeasible. Considering this, we adopt a uniform quantization for the inputs and internal network responses (called feature maps) to maintain low on-chip expenses. The Convolutional Neural Network with reduced weight and input/response precision is demonstrated in recognizing two types of images: one is hand-written digit images and the other is real-life images in office scenarios. Both results show that the new network is able to achieve the performance of the neural network with full bit resolution, even though in the new network the bit resolution of both weight and input are significantly reduced, e.g., from 64 bits to 4-5 bits.
In recent years, sparse coding has drawn considerable research attention in developing feature representations for visual
recognition problems. In this paper, we devise sparse coding algorithms to learn a dictionary of basis functions from Scale-
Invariant Feature Transform (SIFT) descriptors extracted from images. The learned dictionary is used to code SIFT-based
inputs for the feature representation that is further pooled via spatial pyramid matching kernels and fed into a Support
Vector Machine (SVM) for object classification on the large-scale ImageNet dataset. We investigate the advantage of
SIFT-based sparse coding approach by combining different dictionary learning and sparse representation algorithms. Our
results also include favorable performance on different subsets of the ImageNet database.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.