pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

We have witnessed rapid progress on 3D-aware image synthesis, leveraging recent advances in generative visual models and neural rendering. Existing approaches how-ever fall short in two ways: first, they may lack an under-lying 3D representation or rely on view-inconsistent rendering, hence synthesizing images that are not multi-view consistent; second, they often depend upon representation network architectures that are not expressive enough, and their results thus lack in image quality. We propose a novel generative model, named Periodic Implicit Generative Adversarial Networks (π-GAN or pi-GAN), for high-quality 3D-aware image synthesis. π-GAN leverages neural representations with periodic activation functions and volumetric rendering to represent scenes as view-consistent radiance fields. The proposed approach obtains state-of-the-art results for 3D-aware image synthesis with multiple real and synthetic datasets.

1. Introduction

Generative Adversarial Networks (GANs) are capable of generating high-resolution, photorealistic images [24], [25], [26]. However, these GANs are often confined to two dimensions because of a lack of photorealistic 3D training data; therefore, they cannot support tasks such as synthesizing multiple views of a single object. 3D-aware image synthesis offers to learn neural scene representations unsupervised from 2D images. The learned representations can be used to render view-consistent images from new camera poses [41], [54], [18].