An Interpretable Deep Semantic Segmentation Method for Earth Observation

Earth observation is fundamental for a range of human activities including flood response as it offers vital information to decision makers. Semantic segmentation plays a key role in mapping the raw hyper-spectral data coming from the satellites into a human understandable form assigning class labels to each pixel. Traditionally, water index based methods have been used for detecting water pixels. More recently, deep learning techniques such as U-Net started to gain attention offering significantly higher accuracy. However, the latter are hard to interpret by humans and use dozens of millions of abstract parameters that are not directly related to the physical nature of the problem being modelled. They are also labelled data and computational power hungry. At the same time, data transmission capability on small nanosatellites is limited in terms of power and bandwidth yet constellations of such small, nanosatellites are preferable, because they reduce the revisit time in disaster areas from days to hours. Therefore, being able to achieve as highly accurate models as deep learning (e.g. U-Net) or even more, to surpass them in terms of accuracy, but without the need to rely on huge amounts of labelled training data, computational power, abstract coefficients offers potentially game-changing capabilities for EO (Earth observation) and flood detection, in particular. In this paper, we introduce a prototype-based interpretable deep semantic segmentation (IDSS) method, which is highly accurate as well as interpretable. Its parameters are in orders of magnitude less than the number of parameters used by deep networks such as U-Net and are clearly interpretable by humans. The proposed here IDSS offers a transparent structure that allows users to inspect and audit the algorithm’s decision. Results have demonstrated that IDSS could surpass other algorithms, including U-Net, in terms of IoU (Intersection over Union) total water and Recall total water. We used WorldFloods data set for our experiments and plan to use the semantic segmentation results combined with masks for permanent water to detect flood events.