Single-Photon Cameras Image Reconstruction Using Vision Transformer

Single-photon camera is a novel camera type that utilizes image sensor with photon-counting capability. Recently, the potential of such sensors to achieve high spatial resolutions (e.g., 10^9pixels/chip) and frame rates (e.g., 10^6frames/sec) is well-established. However, there is a significant difference in the output data between single-photon cameras and traditional CMOS sensor cameras. Therefore, conventional image reconstruction algorithms cannot be utilized. Vison Transformer has impressive performance on image processing tasks, and this paper will design a plug-and-play algorithm for reconstructing images from single-photon cameras and integrate image processing algorithms based on deep neural networks like ViT. The results demonstrate our methods can achieve improvement both in PSNR and SSIM.

I. Introduction

In terms of single photon cameras, single photon cameras (SPC) are a new type of camera developed based on image sensors with photon counting capabilities, typically referring to photography devices that use single photon avalanche diodes instead of CMOS. The concept of single photon avalanche diode (SPAD) or quantum image sensor was first proposed by Eric Fossum [1]. in 2005 With the continuous development of the semiconductor, cameras based on single photon avalanche diodes have made breakthrough progress in recent years. While in 2017, it could only provide 1 megapixel per chip, these SPAD-based image sensors can now achieve 16.7 megapixels with only 0.12 e-rms best single-pixel read noise [2].