Omniscopic Vision for Robotic Control

Autonomous and semi-autonomous platforms increasingly leverage vision as a reliable, cheap and appropriate sensing method to understand the state of themselves and the surrounding environment. Manufacturers desire to maximize operation capabilities while safeguarding limits to protect the robotic platforms and the environment. Consequently, we want to simultaneously maximize the information we have of the surroundings as well as the confidence of this information, while minimizing the processing complexity. This paper explores the capabilities that high resolution, fully spherical camera systems serve for robotic control. We highlight the effect of camera parameters (sensory and optical) and array layout (sensor count, placement, monoscopic/stereoscopic and stereo disparity) on spatial resolution, Field of View (FoV) and depth estimation. We propose a camera system that uses an array of 16 individually driven cellphone sensors that achieve a combined resolution of up 210 MegaPixels with 360 × 45 degreee coverage. This system demonstrates stereoscopic pairs which serve to easily derive depth, while maintaining a resolving power of 3 cm at 100m with a framerate of up to 30 Hz, equivalent to the human resolving power. The assembly highlights a novel vision capability for ground vehicles, where object detection and odometry is enabled for far-ahead planning and safe operation of vehicles. This paper illustrates design validations for the full system and test results of this camera array. We evaluate system reliability, sensing performance in high dynamic range lighting conditions, and illustrate the data handling of this data-intensive workflow. The physical system is tested in indoor and outdoor scenes with varying light conditions and with both stationary and in motion scenes. Additional array layouts with the same cameras are discussed that expand vertical FoV, explore monoscopic versions and differ in physical footprint.