Toward High-Resolution Face Image Generation From Coded Aperture Camera

Coded aperture cameras can be manufactured cheaply, have a very thin form-factor, and may be transparent and flexible, thus providing easy-to-integrate and compact image sensors. However, limitations in reconstructed image quality and resolution have impeded the growth of their applications. We propose a method to generate high-resolution face images from low-resolution coded aperture sensor snapshots. Using the point spread function of the coded aperture camera, we generate a set of training images to train an image enhancement network. We then apply a face recognition model to extract facial templates and project them into the intermediate latent space of a face generator network to generate high-resolution (i.e., $1024\times 1024$) face images. Our experimental results show significant retention of the subject's identity in the generated high-resolution face images. Our cross-dataset evaluation shows the generalization of our method on other datasets for generating high-resolution face images. To our knowledge, this is the first work for generating high-resolution face images from coded aperture imaging. The source code of our experiments is publicly available to facilitate the reproducibility of our work.

I. Introduction

Coded aperture imaging was initially developed for astronomical imaging in X-ray and gamma-ray wavelengths [1], [2], [3], [4], [5], where optical elements, such as lenses and mirrors, are impossible or prohibitively expensive to manufacture. By their construction, coded aperture (lensless) cameras have several advantages over traditional cameras. Coded aperture masks are far simpler to manufacture than lenses or stacks of lenses, and the flatness of the assembled device results in a more compact package. Furthermore, flexibility and transparency can even be obtained with a selective choice of material [6], allowing integration in bidirectional (sensing and emitting) displays. In addition to visible imaging, the propagation of light through the mask and resulting multiplexing offers the possibility of encoding additional scene content, such as depth or spectral information.