High Frame Rate Video Reconstruction and Deblurring Based on Dynamic and Active Pixel Vision Image Sensor

Active image sensors can only record scenes at low frame rates. The dynamic vision sensor (DVS) can capture high-speed scenes under challenging lighting conditions. However, DVS discards absolute light intensity information. The dynamic and active pixel vision image sensor (DAVIS) can output events and low-frame-rate grayscale images simultaneously. This paper proposes a high frame rate video reconstruction and deblurring algorithm based on events and low-speed blurred image sequences from DAVIS. First, we visualized events as a sequence of event images and performed image denoising process. Then, we used the extended 3D partial recursive search (E-3DPRS) method to generate high-performance motion vector field (MVF) between adjacent denoising event images with low computational complexity. MVF reflects the object motion trajectory with high time resolution. Furthermore, MVF is used as a priori information to restore potentially sharp images, which alleviates the highly ill-posed deblurring process. At last, we reconstructed a high frame rate sharp video based on a sharp image sequence and MVFs. Extensive experiments show that our proposed algorithm outperforms the existing state-of-the-art methods and achieves better average feature similarity and structural similarity.