Convolutional Sparse Coding for Trajectory Reconstruction

Trajectory basis Non-Rigid Structure from Motion (NRSfM) refers to the process of reconstructing the 3D trajectory of each point of a non-rigid object from just their 2D projected trajectories. Reconstruction relies on two factors: (i) the condition of the composed camera & trajectory basis matrix, and (ii) whether the trajectory basis has enough degrees of freedom to model the 3D point trajectory. These two factors are inherently conflicting. Employing a trajectory basis with small capacity has the positive characteristic of reducing the likelihood of an ill-conditioned system (when composed with the camera) during reconstruction. However, this has the negative characteristic of increasing the likelihood that the basis will not be able to fully model the object's “true” 3D point trajectories. In this paper we draw upon a well known result centering around the Reduced Isometry Property (RIP) condition for sparse signal reconstruction. RIP allow us to relax the requirement that the full trajectory basis composed with the camera matrix must be well conditioned. Further, we propose a strategy for learning an over-complete basis using convolutional sparse coding from naturally occurring point trajectory corpora to increase the likelihood that the RIP condition holds for a broad class of point trajectories and camera motions. Finally, we propose an 21 inspired objective for trajectory reconstruction that is able to “adaptively” select the smallest sub-matrix from an over-complete trajectory basis that balances (i) and (ii). We present more practical 3D reconstruction results compared to current state of the art in trajectory basis NRSfM.

1. Introduction

Non-Rigid Structure from Motion (NRSfM) refers to the task of recovering the time varying 3D coordinates of each point on a deforming object from just their 2D projections. One prevalent approach for solving this problem is to seek the 3D reconstruction with the most compact linear shape basis that still satisfies the 2D projections [3]. This strategy works well for short simple sequences containing a single action/event. However, it has inherent problems when dealing with real-world complex motion sequences (e.g., a person walking, jumping, sitting and dancing all in the same sequence) [1]. In these instances the strategy of seeking the most compact linear shape is not sufficient to ensure an accurate reconstruction.