Mr. Jonathan Gonzalez-Santiago Profile

Jonathan Gonzalez-Santiago

at Fraunhofer IOSB

SPIE Involvement:

Author

Publications (3)

Proceedings Article | 26 October 2022 Presentation + Paper

Deep self-supervised band-level learning for hyperspectral classification

Jonathan González Santiago, Fabian Schenkel, Wolfgang Middelmann

Proceedings Volume 12267, 122670I (2022) https://doi.org/10.1117/12.2636245

KEYWORDS: Hyperspectral imaging, Classification systems, Image processing, Image classification, Machine learning

Read Abstract +

Hyperspectral image classification is one of the most researched topics within hyperspectral analysis. Its importance is determined by its immediate outcome, a classified image used for planning and decision-making processes within a variety of engineering and scientific disciplines. Within the last few years, researchers have solved this task employing self-supervised learning to learn robust feature representations to alleviate the dependency on large amounts of labels required by supervised deep learning. Aiming to learn representations for hyperspectral classification purposes, several of these works use dimensionality reduction that could exclude relevant information during feature learning. Moreover, they are based on contrastive instance learning that requires a large memory bank to store the result of pairwise feature discriminations, which represents a computational hurdle. To overcome these challenges, the current approach performs self-supervised cluster assignments between sets of contiguous bands to learn semantically meaningful representations that accurately contribute to solving the hyperspectral classification task with fewer labels. The approach starts with the pre-processing of the data for self-supervised learning purposes. Subsequently, the self-supervised band-level learning phase takes the preprocessed image patches to learn relevant feature representations. Afterwards, the classification step uses the previously learned encoder model and turns it into a pixel classifier to execute the classification with fewer labels than awaited. Lastly, the validation makes use of the kappa coefficient, and the overall and average accuracy as well-established metrics for assessing classification results. The method employs two benchmark datasets for evaluation. Experimental results show that the classification quality of the proposed method surpasses supervised learning and contrastive instance learning-based methods for the majority of the studied data partition levels. The construction of the most adequate set of augmentations for hyperspectral imagery also indicated the potential of the results to further improve.

Proceedings Article | 12 September 2021 Presentation + Paper

Self-supervised multi-task learning for semantic segmentation of urban scenes

Jonathan Gonzalez Santiago, Fabian Schenkel, Wolfgang Middelmann

Proceedings Volume 11862, 118620G (2021) https://doi.org/10.1117/12.2600194

KEYWORDS: Image segmentation, RGB color model, Visualization, Classification systems, Convolution, Remote sensing

Read Abstract +

The task of semantic segmentation plays a vital role in the analysis of remotely sensed imagery. Currently, this task is mainly solved using supervised pre-training, where very Deep Convolutional Neural Networks (DCNNs) are trained on large annotated datasets for mostly solving a classification problem. They are useful for many visual recognition tasks but heavily depend on the amount and quality of the annotations to learn a mapping function for predicting on new data. Motivated by the plethora of data generated everyday, researchers have come up with alternatives such as Self-Supervised Learning (SSL). These methods play a deciding role in boosting the progress of deep learning without the need of expensive labeling. They entirely explore the data, find supervision signals and solve a challenge known as Pretext Task (PTT) to learn robust representations. Thereafter, the learned features are transferred to resolve the so-called Downstream Task (DST), which can represent a group of computer vision applications such as classification or object detection. The current work explores the conception of a DCNN and training strategy to jointly predict on multiple PTTs in order to learn general visual representations that could lead more accurate semantic segmentations. The first Pretext Task is Image Colorization (IC) that identifies different objects and related parts present in a grayscale image to paint those areas with the right color. The second task is Spatial Context Prediction (SCP), which captures visual similarity across images to discover the spatial configuration of patches generated out of an image. The DCNN architecture is constructed considering each particular objective of the Pretext Tasks. It is subsequently trained and its acquired knowledge is transferred into a SSL trunk network to build a Fully Convolutional Network (FCN) on top of it. The FCN with SSL trunk learns a compound of features through fine-tuning to ultimately predict the semantic segmentation. With the aim of evaluating the quality of the learned representations, the performance of the trained model will be compared with inference results of a FCN-ResNet101 architecture pre-trained on ImageNet. This comparison employs the F1-Score as quality metric. Experiments show that the method is capable of achieving general feature representations that can definitely be employed for semantic segmentation purposes.

Proceedings Article | 20 September 2020 Presentation + Paper

Unsupervised hyperspectral classification based on similarity graphs

Jonathan Gonzalez-Santiago, Fabian Schenkel, Wolfgang Gross, Wolfgang Middelmann

Proceedings Volume 11533, 115330S (2020) https://doi.org/10.1117/12.2573660

KEYWORDS: Algorithm development, Image segmentation, Sensors, Spatial resolution, Image classification, Hyperspectral imaging, Seaborgium, Image processing algorithms and systems, Astatine, Analytical research

Read Abstract +

Hyperspectral cluster analysis represents a powerful instrument for land cover classification. It consists of grouping hyperspectral pixels based on a similarity measure that determines the affinity level between data points. Many of the existing clustering methods are not suitable for hyperspectral data due mainly to the socalled curse of dimensionality. The previous fact motivates researchers to develop new clustering algorithms for dealing with high dimensional data. Among these are the techniques based on Spectral Graph Theory (SGT). They regard objects as vertices and their pair-wise similarity as weighted edge to transform the clustering problem into a graph partition task. Their properties make them well-suited for datasets with arbitrary shape and high dimensionality. The current approach strives the unsupervised classification of hyperspectral imagery employing Similarity Graphs (SG). To achieve this goal, a superpixel-based segmentation using the Simple Linear Iterative Clustering (SLIC) algorithm is executed. It takes the input data and groups pixels considering their image proximity and spectral similarity. Subsequently, the superpixels are converted into a Similarity Graph G = (V, E) with vertex set V = V₁, V₂, ..., V_n, where n represents the vertex number. For this conversion, the Adjacency Matrix (AM) is constructed with the similarities between vertices. Consequently, the Laplacian Matrix (LM) is determined to embed the data points into a low-dimensional space. This embedding occurs after finding the eigenvalues and eigenvectors of the LM. At this point, the clustering algorithm groups relevant LM eigenvectors to generate the land cover map. Finally, a comparison between the classified maps and the results of directly applying the Hierarchical Agglomerative Custering (HAC) algorithm on the corresponding superpixels is executed. This analysis considers the correspondence of the results with reality and the magnitude of the Cohen’s Kappa coefficient. The proposed method uses two benchmark datasets to create land cover classification maps. The results show that the method is capable of accurately partitioning data points with moderate overlapping level, where established algorithms such as the HAC still experiences difficulties.

View contact details

UPDATE YOUR PROFILE

Is this your profile? Update it now.

Sign into your SPIE.org account

Don’t have a profile and want one?

Create an account on SPIE.org

Keywords/Phrases

Search In:

Publication Years