KEYWORDS: Virtual reality, Visualization, Data modeling, Data visualization, Computing systems, Scanning transmission electron microscopy, Vegetation, Data analysis, Visual process modeling, 3D modeling
In this paper, we present a newly developed extended reality (XR) environment focused on qualitative and quantitative analysis and visualization of data on deforestation in urban areas and its impact on the area communities. The design and development process followed a user-centric approach that engaged researchers and practitioners. Using off-the-shelf technology such as Meta Quest headsets, the environment was developed in Unity, C#, and Python, and incorporates USGS GIS data layers. Other aspects such as affordability and accessibility were considered by acknowledging individuals with different learning styles and examining a new way to understand data. Compute and storage limitations brought on by the headset were overcome through data sampling and through offloading some of the computing tasks to a separate computer and transmission of the synthesized tasks back to the headset. Initial experiments focused on the ingestion of New York City area data. The region was chosen due to the population density, and the significant socio-economic disparities among various communities, but also due to the availability of ancillary data such as the one provided by the NYC Open Data that can be used to complement the USGS data. Urban and suburban areas were used to find indicators of vegetation and learn about the challenges associated with developing spatial data in different densities. The visualization also showed that while changes in deforestation over the past decade have been fairly uniform in both area types, sub-areas have seen a significant green space decrease. While the current XR environment is envisioned as the first step in the creation of a virtual interactive interface that shows predictive models of urban deforestation, it already constitutes an example of an educational approach to XR development. The code and system description will be made publicly available as Open Source and include mechanisms for community code contributions.
KEYWORDS: Independent component analysis, Hyperspectral imaging, Principal component analysis, Feature extraction, System on a chip, Image classification, Computer science, Multidimensional signal processing, Mining, Sensors
For most of the success, hyperspectral image processing techniques have their origins in multidimensional signal processing with a special emphasis on optimization based on objective functions. Many of these techniques (ICA, PCA, NMF, OSP, etc.) have their basis on collections of single dimensional data and do not take in consideration any spatial based characteristics (such as the shape of objects in the scene). Recently, in an effort to improve the processing results, several approaches that characterize spatial complexity (based on the neighborhood information) were introduced.
Our goal is to investigate how spatial complexity based approaches can be employed as preprocessing techniques for other previously established methods. First, we designed for each spatial complexity based technique a step that generates a hyperspectral cube scaled based on spatial information. Next we feed the new cubes to a group of processing techniques such as ICA and PCA. We compare the results between processing the original and the scaled data. We compared the results on the scaled data with the results on the full data.
We built upon these initial results by employing additional spatial complexity approaches. We also introduced new hybrid approaches that would embed the spatial complexity step into the main processing stage.
KEYWORDS: Data communications, System on a chip, Hyperspectral imaging, Feature extraction, Algorithm development, Distributed computing, Remote sensing, Data storage, Data processing, Solids
A method of incorporating the multi-mixture pixel model into hyperspectral endmember extraction is presented and
discussed. A vast majority of hyperspectral endmember extraction methods rely on the linear mixture model to describe
pixel spectra resulting from mixtures of endmembers. Methods exist to unmix hyperspectral pixels using nonlinear
models, but rely on severely limiting assumptions or estimations of the nonlinearity. This paper will present a
hyperspectral pixel endmember extraction method that utilizes the bidirectional reflectance distribution function to
model microscopic mixtures. Using this model, along with the linear mixture model to incorporate macroscopic
mixtures, this method is able to accurately unmix hyperspectral images composed of both macroscopic and microscopic
mixtures. The mixtures are estimated directly from the hyperspectral data without the need for a priori knowledge of the
mixture types. Results are presented using synthetic datasets, of multi-mixture pixels, to demonstrate the increased
accuracy in unmixing using this new physics-based method over linear methods. In addition, results are presented using
a well-known laboratory dataset.
We present a novel unsupervised method for facial recognition using hyperspectral imaging and decision fusion. In
previous work we have separately investigated the use of spectra matching and image based matching. In spectra
matching, face spectra are being classified based on spectral similarities. In image based matching, we investigated
various approaches based on orthogonal subspaces (such as PCA and OSP). In the current work we provide an
automated unsupervised method that starts by detecting the face in the image and then proceeds to performs both
spectral and image based matching. The results are fused in a single classification decision. The algorithm is tested on an
experimental hyperspectral image database of 17 subjects each with five different facial expressions and viewing angles.
Our results show that the decision fusion leads to improvement of recognition accuracy when compared to the individual
approaches as well as to recognition based on regular imaging.
Nanotechnology is a rapidly emerging field in which the material structures are of the size 100 nanometers or
smaller. Thus, analyzing images at the nanoscale level is a challenging task. Users in this field are interested in image
analysis and processing to draw conclusions such as the impact of various experimental conditions on the nature of the
image and consequently their usefulness in several applications. This motivates our work that involves designing a
system that will not only recognize similarities and differences among images, but do so efficiently and accurately.
Features are representative of the manner in which images are compared by human experts by finding empirical data
about particle sizes, material depth, inter-particle distances and so forth. In this work, we look into the use of features
for comparison by implementing a feature-based algorithm on real image data sets from nanotechnology and thereafter
using the results in processes such as clustering that are commonly applied by users to analyze images. We are able to
effectively assess the feature-based approach in a real-world context as corroborated by our experimental evaluation.
We present an efficient method for facial recognition using hyperspectral imaging and orthogonal subspaces. Projecting
the data into orthogonal subspaces has the advantage of compactness and reduction of redundancy. We focus on two
approaches: Principal Component Analysis and Orthogonal Subspace Projection. Our work is separated in three stages.
First, we designed an experimental setup that allowed us to create a hyperspectral image database of 17 subjects under
different facial expressions and viewing angles. Second, we investigated approaches to employ spectral information for
the generation of fused grayscale images. Third, we designed and tested a recognition system based on the methods
described above. The experimental results show that spectral fusion leads to improvement of recognition accuracy when
compared to regular imaging. The work expands on previous band extraction research and has the distinct advantage of
being one of the first that combines spatial information (i.e. face characteristics) with spectral information. In addition,
the techniques are general enough to accommodate differences in skin spectra.
Feature reduction denotes the group of techniques that reduce high dimensional data to a smaller set of components. In remote sensing feature reduction is a preprocessing step to many algorithms intended as a way to reduce the computational complexity and get a better data representation. Reduction can be done by either identifying bands from the original subset (selection), or by employing various transforms that produce new features (extraction). Research has noted challenges in both directions. In feature selection, identifying an "ideal" spectral band subset is a hard problem as the number of bands is increasingly large, rendering any exhaustive search unfeasible. To counter this, various approaches have been proposed that combine a search algorithm with a criterion function. However, the main drawback of feature selection remains the rather narrow bandwidths covered by the selected bands resulting in possible information loss. In feature extraction, some of the most popular techniques include Principal Component Analysis, Independent Component Analysis, Orthogonal Subspace Projection, etc. While they have been used with success in some instances, the resulting bands lack a physical relationship to the data and are mostly produced using statistical strategies. We propose a new technique for feature reduction that exploits search strategies for feature selection to extract a set of spectral bands from a given imagery. The search strategy uses dynamic programming techniques to identify 'the best set" of features.
Hyperspectral images provide an innovative means for visualizing information about a scene or object that exists outside
of the visible spectrum. Among other capabilities, hyperspectral image data enable detection of contamination in soil,
identification of the minerals in an unfamiliar material, and discrimination between real and artificial leaves in a potted
plant that are otherwise indistinguishable to the human eye. One of the drawbacks of working with hyperspectral data is
that the massive amounts of information they provide requiring efficient means of being processed. In this study wavelet
analysis was used to approach this problem by investigating the capabilities it provides for producing a visually
appealing image from data that have been reduced in the spatial and spectral dimensions. We suggest that a procedure
for visualizing hyperspectral image data that uses the peaks of the spectral signatures of pixels of interest provides a
promising method for visualization. Using wavelet coefficients and data from the hyperspectral bands produces
noticeably different results, which suggests that wavelet analysis could provide a superior means for visualization in
some instances when the use of bands does not provide acceptable results.
Among various biometrics measures used in human identification, face recognition, has the distinct advantage of not
requiring the subjects collaboration. Hyperspectral data constitute a natural choice for expanding face recognition image
fusion, especially since it may provide information beyond the normal visible range, thus exceeding the normal human
sensing. In this paper we investigate algorithms that improve face recognition by extracting the 'best bands' according to
various criteria such as decorrelation and statistical independence. The work expands on previous band extraction results
and has the distinct advantage of being one of the first that combines spatial information (i.e. face characteristics) with
spectral information.
We describe a novel approach to produce color composite images from hyperspectral data using weighted spectra averages. The weighted average is based on a sequence of numbers (weights) selected using pixel value information and interband distance. Separate sequences of weights are generated for each of the three color bands forming the color composite image. Tuning of the weighting parameters and emphasis on different spectral areas allows for emphasis of one or other feature in the image. The produced image is a distinct approach from a regular color composite result, since all the bands provide information to the final result.
The algorithm was implemented in high level programming language and provided with a user friendly graphical interface. The current design allows for stand-alone usage or for further modifications into a real time visualization module. Experimental results show that the weighted color composition is an extremely fast visualization tool.
Face recognition continues to meet significant challenges in reaching accurate results and still remains one of the activities where humans outperform technology. An attractive approach in improving face identification is provided by the fusion of multiple imaging sources such as visible and infrared images. Hyperspectral data, i.e. images collected over hundreds of narrow contiguous light spectrum intervals constitute a natural choice for expanding face recognition image fusion, especially since it may provide information beyond the normal visible range, thus exceeding the normal human sensing. In this paper we investigate the efficiency of hyperspectral face recognition through an in house experiment that collected data in over 120 bands within the visible and near infrared range. The imagery was produced using an off the shelf sensor in both indoors and outdoors with the subjects being photographed from various angles. Further processing included spectra collection and feature extraction. Human matching performance based on spectral properties is discussed.
We investigate the use of a flexible grid architecture for hyperspectral image processing. Recording data in tens or hundreds of narrow contiguous spectral intervals, hyperspectral data outperform multispectral imagery by allowing the detection of relatively small differences in material composition and of targets occupying a surface smaller than the one covered by a pixel (called subpixel targets). However, with increased spatial and spectral resolution, processing such data often leads to computational costs prohibitive to regular computer systems. While distributed or parallel computing are often found as solutions, many current configurations are still unable to reach the computational complexity level that is required for exhaustive search solutions. In this environment, grid computing becomes a viable alternative. Grid computing, an emerging computing model, is based on the concept of distributing processes across a parallel infrastructure. Throughput is further increased by networking many heterogeneous resources across administrative boundaries to model a virtual computer architecture. Compared to distributed clusters or parallel machines, grid systems are often inexpensive or even free since they can consist of non-dedicated computer systems that are underutilized and have extra CPU cycles that can be spared.
We present general considerations on grid architectures and discuss the current grid environment we have deployed. Next, we investigate exhaustive band search, a data processing problems that suffers from large computational requirements and present our grid based solutions for it. Our experimental results indicate a significant speedup in obtaining results and even solving of problems otherwise not tractable in regular computing environments.
We present a new algorithm for feature extraction in hyperspectral images based on source separation and parallel computing. In source separation, given a linear mixture of sources, the goal is to recover the components by producing an unmixing matrix. In hyperspectral imagery, the mixing transform and the separated components can be associated with endmembers and their abundances. Source separation based methods have been employed for target detection and classification of hyperspectral images. However, these methods usually involve restrictive conditions on the nature of the results such as orthogonality (in Principal Component Analysis - PCA and Orthogonal Subspace Projection - OSP) of the endmembers or statistical independence (in Independent Component Analysis - ICA) of the abundances nor do they fully satisfy all the conditions included in the Linear Mixing Model. Compared to this, our approach is based on the Nonnegative Matrix Factorization (NMF), a less constraining unmixing method. NMF has the advantage of producing positively defined data, and, with several modifications that we introduce also ensures addition to one. The endmember vectors and the abundances are obtained through a gradient based optimization approach. The algorithm is further modified to run in a parallel environment. The parallel NMF (P-NMF) significantly reduces the time complexity and is shown to also easily port to a distributed environment. Experiments with in-house and Hydice data suggest that NMF outperforms ICA, PCA and OSP for unsupervised endmember extraction. Coupled with its parallel implementation, the new method provides an efficient way for unsupervised unmixing further supporting our efforts in the development of a real time hyperspectral sensing environment with applications to industry and life sciences.
Hyperspectral data is modeled as an unknown mixture of original features (such as the materials present in the scene). The goal is to find the unmixing matrix and to perform the inversion in order to recover them. Unlike first and second order techniques (such as PCA), higher order statistics (HOS) methods assume the data has nongaussian behavior are able to represent much subtle differences among the original features. The HOS algorithms transform the data such that the result components are uncorrelated and their nongaussianity is maximized (the resulting components are statistical independent). Subpixel targets in a natural background can be seen as anomalies of the image scene. They expose a strong nongaussian behavior and correspond to independent components leading to their detection when HOS techniques are employed. The methods start by preprocessing the hyperspectral image through centering and sphering. The resulting bands are transformed using gradient-based optimization on the HOS measure. Next, the data are reduced through a selection of the components associated with small targets using the changes of the slope in the scree graph of the non-Gaussianity values. The targets are filtered using histogram-based analysis. The end result is a map of the pixels associated with small targets.
This paper describes a new algorithm for feature extraction on hyperspectral images based on blind source separation (BSS) and distributed processing. I use Independent Component Analysis (ICA), a particular case of BSS, where, given a linear mixture of statistical independent sources, the goal is to recover these components by producing the unmixing matrix. In the multispectral/hyperspectral imagery, the separated components can be associated with features present in the image, the source separation algorithm projecting them in different image bands. ICA based methods have been employed for target detection and classification of hyperspectral images. However, these methods involve an iterative optimization process. When applied to hyperspectral data, this iteration results in significant execution times. The time efficiency of the method is improved by running it on a distributed environment while preserving the accuracy of the results. The design of the distributed algorithm as well as issues related to the distributed modeling of the hyperspectral data were taken in consideration and presented. The effectiveness of the proposed algorithm has been tested by comparison to the sequential source separation algorithm using data from AVIRIS and HYDICE. Preliminary results indicate that, while the accuracy of the results is preserved, the new algorithm provides a considerable speed-up in processing.
This paper investigates the effect of spectral screening on processing hyperspectral data through Independent Component Analysis (ICA). ICA is a multivariate data analysis method producing components that are statistically independent. In the context of the linear mixture model, the endmember abundances can be viewed as independent components, the endmembers forming the columns of the mixing matrix. In essence, the ICA processing can be seen as an alternative solution to endmember unmixing. In the context of feature extraction, each feature will be represented by an independent component, thus leading to maximum separability among features. Spectral screening is defined as the reduction of the image cube to a subset of representative pixel vectors with the goal of achieving a considerable speedup in further processing. At the base of spectral screening are the measure used to assess the similarity between two pixel vectors and a threshold value. Two pixel vectors are similar if the value yielded by the similarity measure is smaller than the threshold and dissimilar otherwise. The spectral screened subset has to be formed such that any two vectors in the subset are dissimilar and for any vector in the original image cube there is a similar vector in the subset. The method we present uses spectral angle as distance measure. A necessary condition for the success of spectral screening is that the result of processing the reduced subset can be extended to the entire data. Intuitively, a larger subset would lead to increased accuracy. However, the overhead introduced by the spectral screening is directly proportional to the subset size. Our investigation has focused on finding the “ideal” threshold value that maximizes both the accuracy and the speedup. The practical effectiveness of the methods was tested on HYDICE data. The results indicate that considerable speedup is obtained without a considerable loss of accuracy. The presented method leads to a significant increase in computational efficiency allowing faster processing of hyperspectral images.
The paper presents a novel algorithm based on Independent Component Analysis (ICA) for the detection of small targets present in hyperspectral images. Compared to previous approaches, the algorithm provides two significant improvements. First, an important speedup is obtained by preprocessing the data through spectral screening. Spectral screening is a technique that measures the similarity between pixel vectors by calculating the angle between them. For a certain threshold α, a set of pixel vectors is selected such that the angle between any two of them is larger than α and the angle between any of the pixel vectors not selected and at least one selected vector is smaller than α. In addition to significantly reducing the size of the data, spectral screening reduces the influence of dominating features. The second improvement is the modification of the Infomax algorithm such that the number of components that are produced is lower than the number of initial observations. This change eliminates the need for feature reduction through PCA, and leads to increased accuracy of the results. Results obtained by applying the new algorithm on data from the hyperspectral digital imagery collection experiment (HYDICE) show that, compared with previous ICA based target detection algorithms developed by the authors, the novel approach has an increased efficiency, at the same time achieving a considerable speedup. The experiments confirm the efficiency of ICA as an attractive tool for hyperspectral data processing.
The paper presents an algorithm based on Independent Component Analysis (ICA) for the detection of small targets present in hyperspectral images. ICA is a multivariate data analysis method that attempts to produce statistically independent components. This method is based on fourth order statistics. Small, man-made targets in a natural background can be seen as anomalies in the image scene and correspond to independent components in the ICA model. The algorithm described here starts by preprocessing the hyperspectral data through centering and sphering, thus eliminating the first and second order statistics. It then separates the features present in the image using an ICA based algorithm. The method involves a gradient descent minimization of the mutual information between frames. The resulting frames are ranked according to their kurtosis (defined by normalized fourth order moment of the sample distribution). High kurtosis valued frames indicate the presence of small man-made targets. Thresholding the frames using zero detection in their histogram further identifies the targets. The effectiveness of the method has been studied on data from the hyperspectral digital imagery collection experiment (HYDICE). Preliminary results show that small targets present in the image are separated from the background in different frames and that information pertaining to them is concentrated in these frames. Frame selection using kurtosis and thresholding leads to automated identification of the targets. The experiments show that the method provides a promising new approach for target detection.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.