Cross-Frame Foreground Structural Similarity Modeling by Convolutional Sparse Representation

In this paper, we propose a foreground-background separation (FBS) method using a novel foreground model based on convolutional sparse representation (CSR). Existing FBS methods are based on the assumption that the foreground component is sparse or continuous in time, resulting in poor separation performance for videos containing sparse noise/fast moving objects or for low frame rate ones. Therefore, we propose to use CSR to characterize the similar structure of the foreground component scattered in each frame. This CSR-based model can adaptively capture the "shape" of the foreground component and thus can accurately separate it even under the above difficulties. We formulate FBS as an optimization problem incorporating the CSR-based model and develope an algorithm to solve it based on alternating minimization. Experiments using an infrared video and an electron microscope video show that the proposed foreground model is superior to existing models.

I. Introduction

Foreground-background separation (FBS) is a preprocessing technique for decomposing video data into dynamic foreground and static background components [1]. These components contain helpful information for subsequent processing, such as motion detection [2], moving object detection [3], and background subtraction [4]. However, since video data (especially those obtained from specialized measurements such as infrared videos and microscope videos) often contains various types of noise, it is very important to establish noise-robust FBS techniques.