An FPGA-Accelerated Ultra-High-Speed Stereo Rectification for Event-Based Camera

Event-cameras are bio-inspired vision sensors, which sense brightness changes and provide a latency in the order of micro-seconds. Recently, some researchers have studied on efficient event-camera reconstruction approaches to obtain high-quality images. These efforts make performing stereo vision based on event-cameras being possible. Stereo rectification, as an important procedure of stereo vision, is mainly used to align stereo images captured by the stereo camera and remove the distortion of the lens. However, traditional stereo rectification architecture cannot process such high-speed image streams generated by the event-camera in real-time, which calls for new approaches. In this paper, we propose an FPGA-accelerated ultra-high-speed stereo rectification architecture to calibrate the images generated by event-cameras. In this architecture, a fully pipelined calculation module is designed to process the complex matrix transformation operations, which puts in significant pipelining registers to maximize the frequency. To further improve the throughput, we propose a parallel processing architecture that uses multiple processing elements (PEs) to process multiple pixels per cycle. In addition, we design a memory management unit to optimize the memory usage of the hardware resource. The whole rectification architecture is effectively implemented on a Xilinx Zynq7100 FPGA. The experiments show that our architecture can process a pair of 400×250 images generated by the event-cameras at a speed of 8800 frames per second.