Attribution Modeling for Deep Morphological Neural Networks using Saliency Maps

Mathematical morphology has been explored in deep learning architectures, as a substitute to convolution, for problems like pattern recognition and object detection. One major advantage of using morphology in deep learning is the utility of morphological erosion and dilation. Specifically, these operations naturally embody interpretability due to their underlying connections to the analysis of geometric structures. While the use of these operations results in explainable learned filters, morphological deep learning lacks attribution modeling, i.e., a paradigm to specify what areas of the original observed image are important. Furthermore, convolution-based deep learning has achieved attribution modeling through a variety of neural eXplainable Artificial Intelligence (XAI) paradigms (e.g., saliency maps, integrated gradients, guided backpropagation, and gradient class activation mapping). Thus, a problem for morphology-based deep learning is that these XAI methods do not have a morphological interpretation due to the differences in the underlying mathematics. Herein, we extend the neural XAI paradigm of saliency maps to morphological deep learning, and by doing, so provide an example of morphological attribution modeling. Furthermore, our qualitative results highlight some advantages of using morphological attribution modeling.

I. Introduction

Deep learning has become one of the largest contributions of the computer vision community due to its quantitative success across tasks like pattern recognition and object detection [1]–[4]. This success is often attributed to the filter hierarchy of feature extraction found in most Convolutional Neural Networks (CNNs), i.e., the convolution operation finds low level features (e.g. lines, edges, etc.) in the beginning layers and combines these features to learn complex high level features (e.g. common objects, shapes, etc.) in deeper layers. However, while convolution based deep learning is renown for producing state of the art quantitative results, other mathematical paradigms such as morphology are becoming valid alternatives - especially when explainability is an important criteria [5], [6]. Morphology embodies characteristics of topology, set theory, and lattice theory for the analysis and processing of geometric structures [7]–[10]. Morphology in practice heavily utilizes both erosion and dilation to calculate the minimal offset by which the foreground and background of a target pattern fits in an image [10]. Furthermore, these operations can be combined to create more complex operations like opening, closing, and hit-or-miss transform [7], [10]. The morphological operations like erosion and dilation have the following advantages over standard convolution. First, erosion and dilation explicitly analyzes both foreground and background information when evaluating the shape of a target class [7], [10]. Second, the filters learned by these operations are more interpretable such that one can visualize the shapes that drove a machine to make its decision. Because of these reasons and more, morphology has been explored in deep learning architectures, as a substitute to convolution, for problems like pattern recognition and object detection [11]–[15]. However, when faced with the task of attribution modeling, morphological deep learning does not have a straightforward approach. Convolution based deep learning, on the other hand, has facilitated attribution modeling through a variety of neural paradigms [16]–[18]. Initially, these techniques only served to highlight what features of the original image were most important for a trained model to perform a specific task. But as the field of XAI flourished, these techniques have become a foundation of visual analytics or graphical explanations for the deep learning community.