Graph Convolutional Network (GCN) is a graph-based learning framework that goes beyond the limitations of regular grid-sampled convolution operated by traditional CNN. By incorporating graph topological structure to the network, GCN is capable of encoding and aggregating features from neighboring nodes and exploiting the data nature in a non-linear way. For HSI data of cultural heritage artifacts, the lack of ground-truth labels imposes extra difficulty on the hyperspectral image classification task using a learning framework. For differentiating subtle spectral differences between varied pigments, we develop a two-layer semi-supervised GCN model for pigment classification, which can be directly applied to an adjacent region of interest (ROI) for automatically clustering unseen pixels without ground-truth labels. We first explore graph construction approaches in depth, and focus on two advanced graph construction strategies: Density-weighted kNN and Natural Nearest Neighbor (NNN). By assigning adaptive number of neighbors based on local data density, a local-scale adaptive graph structure can be built and infused into the GCN model. The experiments on the Selden Map and the Gough Map prove the validity of our method. Under semi-supervised learning framework, SAD+DWkNN GCN model can achieve near 90% classification accuracy for each class based on less than 30% labeled training samples. Compared to the ground-truth labels produced by the Graph Modularity algorithm, the GCN classification map matches even better to the actual spectral characteristics of each material via spectral analysis. The learning and generalization ability of the proposed GCN model shows a promising prospect for applying Graph Convolutional Network for a large-scale HSI clustering task. This research can alleviate the dilemma of deficient labeled samples and aid historians for analyzing varied pigments composition in a more intelligent manner.
We present an automatic clustering algorithm for hyperspectral imagery of cultural heritage artifacts: the Selden Map and the Gough Map, both medieval artifacts imaged in the collections at the Bodleian Library of Oxford University. Unlike "traditional" remotely sensed hyperspectral data, these images are of man-made objects using specific materials meant to visually show feature differences and similarities. Consequently, the data are inherently non-Gaussian and potentially very non-linear in the spectral domain. First, we explore the effective graph representations for hyperspectral images, then optimally select the graph modularity to find community structures for a ROI within the scene. By utilizing the eigenvector of the modularity matrix associated with the largest positive eigenvalue for group labeling, we recursively identify multi-level subgroups existing in the graph, producing a variable level of detail cluster-map based on a cluster tree strategy. The generated non-linear decision boundaries are allowed to take any shape with no limits to cluster size. The clustering metric is determined by optimally selecting a high modularity and the largest positive eigenvalue, as well as considering the magnitude of the entry in the leading eigenvector to make each division more accurate and robust. As a result, the optimal number of clusters are found to best characterize the data. Compared to the traditional clustering methods, such as K-means, the graph modularity-based method can extract perceptually important non-local properties of an object, thus yielding semantically more meaningful cluster groups and better discriminating subtle spectral differences between varied pigments. For the Selden Map, we investigate subtle differences in black inks used to denote navigation routes. For the Gough Map, we look at a specific feature under investigation by historians: the castle depicting London. Our results demonstrate the effectiveness of the method as the clustering results explain and match the actual spectra well. This research aims to aid historians in analysis of pigment composition and further facilitate the study in inference of the creation and the evolving timeline of these artifacts.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.