Detecting 3-D Mirror Symmetry in a 2-D Camera Image for 3-D Shape Recovery

In this paper, we take up the long-standing problem of how to recover 3-D shapes represented by a 2-D image, such as the image on the retina of the eye, or in a video camera. Our approach is biologically grounded in a theory of how the human visual system solves this problem, focusing on shapes that are mirror symmetrical in 3-D. A 3-D mirror-symmetrical shape can be recovered from a single 2-D orthographic or perspective image by applying several a priori constraints: 3-D mirror symmetry, 3-D compactness, and planarity of contours. From the computational point of view, the application of a 3-D symmetry constraint is challenging because it requires establishing 3-D symmetry correspondence among features of a 2-D image, which itself is asymmetrical for almost all viewing directions relative to the 3-D symmetrical shape. We describe new invariants of a 3-D to 2-D projection for the case of a pair of mirror-symmetrical planar contours, and we formally state and prove the necessary and sufficient conditions for detection of this type of symmetry in a single orthographic and perspective image.

I. INTRODUCTION

This paper is motivated by our recent discovery of the fundamental role of symmetry in human 3-D vision. Before we explain why and how symmetry is used by the human visual system, we state what we call a fundamental conjecture of human vision: the human visual system produces 3-D veridical interpretations of 3-D natural shapes and scenes based on the information obtained from 2-D retinal images. By “veridical” we mean that we see things the way they are “out there.” This conjecture makes our approach to vision completely different from what most, if not all, of the human and machine vision researchers have been doing for years. As such, our conjecture represents a paradigm shift in vision [21]. Our paradigm shift is represented by assigning a central role to the following three critical features that are absent or downplayed in other approaches: 1) 3-D recovery (see [2], for a recent history of computer vision, which points out the absence of 3-D recovery in this field, and [25], describing the current mainstream of computer vision, in which 3-D recovery is assigned a secondary role, at best); 2) veridicality of vision (see [18], [26], and [31] for widely discussed and accepted arguments against perceptual veridicality); and 3) predilections (a priori constraints) that are built into the visual system (see the recent review of work on shape perception [3], which makes it clear that visual data completely dominate the current explanations and theories of human and machine vision).