Occlusion boundary detection and figure/ground assignment from optical flow

In this work, we propose a contour and region detector for video data that exploits motion cues and distinguishes occlusion boundaries from internal boundaries based on optical flow. This detector outperforms the state-of-the-art on the benchmark of Stein and Hebert, improving average precision from .58 to .72. Moreover, the optical flow on and near occlusion boundaries allows us to assign a depth ordering to the adjacent regions. To evaluate performance on this edge-based figure/ground labeling task, we introduce a new video dataset that we believe will support further research in the field by allowing quantitative comparison of computational models for occlusion boundary detection, depth ordering and segmentation in video sequences.

1. Introduction

Vision systems make use of optical flow for a number of purposes, such as egomotion estimation and scene structure recovery, the latter including both metric depth estimates and ordinal relationships like figure/ground. In this paper, we focus particularly on the role of motion for grouping and figure/ground assignments. The importance of motion cues in these tasks is a classic point in the psychophysical literature. Koffka stated the Gestalt principle of “common fate” where similarly moving points are perceived as coherent entities [15], and grouping based on motion was emphasized by numerous other works including Gibson [12], who also pointed out occlusion/disocclusion phenomena. In contrast to color and stereopsis, which also help to separate different objects, motion is a cue shared by basically all visual species - a fact that emphasizes its importance in biological vision systems. Figure 1. <bold>Occlusion boundary detection benchmark</bold>. Precision-recall curves for the occlusion boundary detection task reported by Stein and Hebert [24]. We show results reported by [24], Sargin et al. [21], He et al. [14] as well as our own results. For performance numbers, see Table 1.