A Unified Rolling Shutter and Motion Blur Model for 3D Visual Registration

Motion blur and rolling shutter deformations both inhibit visual motion registration, whether it be due to a moving sensor or a moving target. Whilst both deformations exist simultaneously, no models have been proposed to handle them together. Furthermore, neither deformation has been considered previously in the context of monocular full-image 6 degrees of freedom registration or RGB-D structure and motion. As will be shown, rolling shutter deformation is observed when a camera moves faster than a single pixel in parallax between subsequent scan-lines. Blur is a function of the pixel exposure time and the motion vector. In this paper a complete dense 3D registration model will be derived to account for both motion blur and rolling shutter deformations simultaneously. Various approaches will be compared with respect to ground truth and live real-time performance will be demonstrated for complex scenarios where both blur and shutter deformations are dominant.

1. Introduction

Electronic rolling shutter (RS) cameras have been becoming increasingly present in a wide number of applications and devices due to their low cost, low power consumption and continual read-out properties. In particular they are able to acquire much higher frequency scene dynamics via their intrinsic time-varying intra-image measurements where as GS sensors acquire the entire image at the same time instant. This comes, however, at the cost of a more complex camera projection model. More specifically, each horizontal scanline in a RS sensor is acquired at a different time instant and the data can be read-out in parallel. Unfortunately RS cameras capture deformed images if the camera is in motion or objects move in the scene.