Using Hierarchical Superpixels to Improve Neural Network Attributions for Geospatial Models

Abstract

Advances in machine learning (ML) are enabling the development of sophisticated AI models capable of representing highly nonlinear systems. These are increasingly used to model geophysical systems, but the complexity of these models makes it difficult to determine what they learned. Exposing what models have learned may provide opportunities for verifying that they rely on physically meaningful strategies, identifying debugging opportunities, and possibly revealing novel insights into real-world geophysical systems. The opaque nature of complex ML models motivated the rapidly growing field of explainable artificial intelligence (XAI) techniques that attempt to reveal the inner workings of ML models. Here, we focus on attribution methods that capture each feature's contribution to a particular model prediction. For geospatial data, each grid cell represents a model input feature. Correlated input features can make it challenging to use XAI methods to learn about either the model or the underlying data relationships. Geophysical models commonly use gridded geospatial data (e.g. satellite imagery) characterized by extensive correlation. This research reveals major challenges in explaining models that use complex geospatial data. To mitigate these issues, we propose a hierarchical attribution approach to analyze the learned features across multiple scales. We demonstrate that this approach can greatly enhance model interpretability. In the first phase of the investigation, we develop synthetic benchmarks to analyze how strongly correlated inputs affect attributions for multilayer perceptron networks. These benchmarks are designed so that the ground truth attribution is known, enabling quantitative XAI evaluation. We show that correlated features enable models to learn a variety of functions, leading to large disagreements with the ground truth. We then show that these disagreements are largely due to strong autocorrelation. While XAI methods applied to explain individual pixels yielded inconsistent explanations, XAI applied to larger superpixel scales substantially improved the agreement among attributions. In the second phase, more complex datasets are used to train convolutional neural networks capable of learning large-scale spatial patterns. Since the models learn spatial features, we generate a hierarchy of attributions from fine to coarse resolution. Since single pixels are not informative in isolation, applying attribution methods to explain individual grid cells may not reveal the spatial patterns that drive the model. Instead, attribution methods can be applied to groups of grid cells. We demonstrate that the partitioning of the input into grouped features has a considerable impact on the attributions. Very often, our results show that the grid cell-level attributions are noisy and fail to reveal the spatial features that the model relies on. After exposing this issue, we propose strategies using hierarchical clustering of the input data to generate superpixels to achieve more useful XAI results. Our results highlight that XAI methods can produce very misleading explanations. However, we propose strategies to detect that these issues are occurring as well as potential solutions based on training multiple models and applying attribution methods to a hierarchy of grid cell clusters to identify the spatial patterns that the model uses.