Static-Scene Constrained Optimization for Matrix/Tensor-Decomposition-free Foreground-Background Separation

We propose an efficient foreground-background separation (FBS) method for (possibly noisy) video data. Most existing FBS methods model the background as a low-rank component. However, this approach is computationally expensive because it requires matrix/tensor decomposition of high-dimensional videos. In this paper, we first introduce a new background model, named static scene constraint (SSC), to FBS. SSC plays a role in accurately capturing the static background by keeping the temporal gradient of the background component to zero. In addition, SSC is formulated as a convex constraint using differences in the temporal direction, which eliminates the need for matrix/tensor decomposition in optimization and significantly reduces the computational cost compared to existing low-rank-based background models. Second, we formulate the FBS problem as a convex optimization problem involving SSC and develop an efficient solver based on a preconditioned primal-dual splitting algorithm, which can automatically determine the appropriate stepsizes based on problem structure. Finally, we demonstrate the efficiency and effectiveness of our method compared with state-of-the-art FBS methods through experiments using infrared and electron microscope videos.

1. Introduction

Foreground-background separation (FBS), which decomposes video data into foreground and background components [1], is a helpful video preprocessing technique for subsequent processing, such as motion detection [2], moving object detection [3], and background subtraction [4]. However, since video data (especially those obtained from specialized measurements such as thermal infrared videos and microscope videos) often contains various types of noise, it is very important to establish noise-robust FBS techniques.