Continuous procrustes analysis to learn 2D shape models from 3D objects

Two dimensional shape models have been successfully applied to solve many problems in computer vision such as object tracking, recognition and segmentation. Typically, 2D shape models (e.g. Point Distribution Models, Active Shape Models) are learned from a discrete set of image landmarks once the rigid transformations are removed applying Procrustes Analysis (PA). However, the standard PA process suffers from two main limitations: (i) the 2D training samples do not necessarily cover a uniform sampling of all 3D transformations of an object. This can bias the estimate of the shape model; (ii) it can be computationally expensive to learn the shape model by sampling 3D transformations; To solve these problems, we propose Continuous Procrustes Analysis (CPA). CPA uses a continuous formulation that avoids the need to generate 2D projections from all 3D rigid transformations. Furthermore, it builds an efficient (space and time) non-biased 2D shape model from a 3D model of an object. Preliminary experimental results to build 2D shape models of objects and faces show the benefits of CPA over PA.

1. Introduction

Shape models have been successfully applied to solve many computer vision problems such as: object recognition [22], [13], face tracking [1], [7], [2], [3], [9], and image segmentation [19], [17]. The theoretical validity for the use of 2D shape models was proved by Ullman [22] and Tomasi/Kanade [21], who showed how different 2D views of a 3D object can be recovered with a three dimensional subspace (under orthographic projection). This fact has been used in many computer vision algorithms to learn shape models by labeling the projections of 3D objects. Point Distribution Models (PDMs) and Active Shape Models (ASMs) [4] are among the most popular techniques that use 2D shape models. PDMs and ASMs build a shape model from a two dimensional training set of landmarks (projections from a finite set of 3D points on a shape surface). Procrustes Analysis (PA) [10] is used to remove rigid transformations and Principal Component Analysis (PCA) is applied to construct a subspace that models the variation of the normalized shapes [4]. Figure 1 (top) illustrates the PA process: given a 3D object, PA computes the 2D mean shape that after rigid transformations (e.g. Euclidean or affine) can better reconstruct (in the least-squares sense) the projections of the object from different views. Although PA has been extensively used, it suffers from several limitations: (i) the 2D training samples do not necessarily cover a uniform sampling of all 3D transformations of an object. This can bias the estimate of the 2D models towards some particular configurations; (ii) it is computationally expensive to compute 2D projections from all possible 3D trans-formations of an object; (iii) PA aligns the samples w.r.t. to the mean to remove rigid transformations. The residual variation of the shape (difference w.r.t the mean) is modeled with unsupervised algorithms (e.g. PCA). Independently estimating the registration parameters and the model shape parameters might result in a loss of relevant information. Figure 1 Illustration of Procrustes Analysis (PA) (top) and Con-tinuous Procrustes Analysis (CPA) (bottom) to construct 2D shape models.