NeRF-SR++: Towards Higher Quality Supersampled Neural Radiation Fields

Super-resolution combined with novel image synthesis is an advanced image processing method to synthesize low-resolution images into new high-resolution images. NeRF-SR is the first model to obtain decent multi-view super-resolution results with only low-resolution input images, but the super-sampling method implemented using the original Nerf’s MLP network cannot represent the complex details of the scene well. We consider that the volume density and color features obtained by the MLP network do not take into account the global geometry along the ray and the color relationship between the sampling points. To tackle this challenge, we introduce an attention-based model and auto-encoding network to synthesize high-fidelity views from low-resolution input to high-resolution output. The attention-based model mixes the pixel color information of the sampling points on each ray and supervises using ground-truth colors. At the same time, the auto-encoding network learns the global geometry along the ray. Experimental results demonstrate that our model can produce high-quality results for high-resolution new view synthesis, both on synthetic and real-world datasets.