An Effective Missing-Data Estimation Approach for Small-Size Image Sequences

Missing data is a frequently encountered problem for structure-from-motion (SFM) where the 3D structure of an object is estimated based on 2D images. In this paper, an effective approach is proposed to deal with the missing-data estimation problem for small-size image sequences. In the proposed method, a set of sub-sequences is first extracted. Each sub-sequence is composed of the frame to be estimated and a part of the original sequence. In order to obtain diversified estimations, multiple weaker estimators are constructed by means of the column-space-fitting (CSF) algorithm. The various sub-sequences are in turn used as the inputs to the algorithm. As the non-missing entries are known, the estimation errors of these entries are computed so as to select weaker estimators with better estimation performances. Furthermore, a linear programming based weighting model is established to compute the weights for the selected weaker estimators. After the weighting coefficients are obtained, a linear weighting estimation which is used as the final estimation of the missing entries is computed based on the outputs of the weaker estimators. By applying the strategies of weaker-estimator selection and the linear programming weighting model, the proposed missing entry estimation method is more accurate and robust than the existing algorithms. Experimental results on several widely used image sequences demonstrate the effectiveness and feasibility of the proposed algorithm.

I. Introduction

Structure-from-motion (SFM) is the process of recovering the relative camera motion, and the time-varying 3D coordinates of feature points on a deformed or non-deformed object, by means of the corresponding 2D points in a sequence of images [1]. In many cases [2]–[4], the recovered 3D shapes can effectively enhance the performances of existing systems in the areas of object recognition, face perception, etc. As a very important technique, SFM has been studied extensively in the field of computer vision.