Deep Watershed Transform for Instance Segmentation

Most contemporary approaches to instance segmentation use complex pipelines involving conditional random fields, recurrent neural networks, object proposals, or template matching schemes. In this paper, we present a simple yet powerful end-to-end convolutional neural network to tackle this task. Our approach combines intuitions from the classical watershed transform and modern deep learning to produce an energy map of the image where object instances are unambiguously represented as energy basins. We then perform a cut at a single energy level to directly yield connected components corresponding to object instances. Our model achieves more than double the performance over the state-of-the-art on the challenging Cityscapes Instance Level Segmentation task.

1. Introduction

Instance segmentation seeks to identify the semantic class of each pixel as well as associate each pixel with a physical instance of an object. This is in contrast with semantic segmentation, which is only concerned with the first task. Instance segmentation is particularly challenging in street scenes, where the scale of the objects can vary tremendously. Furthermore, the appearance of objects are affected by partial occlusions, specularities, intensity saturation, and motion blur. Solving this task will, however, tremendously benefit applications such as object manipulation in robotics, or scene understanding and tracking for self-driving cars.