Extending the Morphological Hit-or-Miss Transform to Deep Neural Networks

While most deep learning architectures are built on convolution, alternative foundations such as morphology are being explored for purposes such as interpretability and its connection to the analysis and processing of geometric structures. The morphological hit-or-miss operation has the advantage that it considers both foreground information and background information when evaluating the target shape in an image. In this article, we identify limitations in the existing hit-or-miss neural definitions and formulate an optimization problem to learn the transform relative to deeper architectures. To this end, we model the semantically important condition that the intersection of the hit and miss structuring elements (SEs) should be empty and present a way to express Don’t Care (DNC), which is important for denoting regions of an SE that are not relevant to detecting a target pattern. Our analysis shows that convolution, in fact, acts like a hit-to-miss transform through semantic interpretation of its filter differences. On these premises, we introduce an extension that outperforms conventional convolution on benchmark data. Quantitative experiments are provided on synthetic and benchmark data, showing that the direct encoding hit-or-miss transform provides better interpretability on learned shapes consistent with objects, whereas our morphologically inspired generalized convolution yields higher classification accuracy. Finally, qualitative hit and miss filter visualizations are provided relative to single morphological layer.

I. Introduction

Deep learning has demonstrated robust predictive accuracy across a wide range of applications. Notably, it has achieved and, in some cases, surpassed human-level performance in many cognitive tasks, for example, object classification, detection, and recognition, semantic and instance segmentation, and depth prediction. This success can be attributed in part to the ability of a neural network (NN) to construct an arbitrary and very complex function by composition of simple functions, thus empowering it as a formidable machine learning tool.