Pseudo Pansharpening NeRF for Satellite Image Collections

The use of NeRF to model 3D scenes from satellite images is becoming increasingly common. However, the models proposed to date assume the availability of pre-processed RGB images as input. This contrasts with the multispectral nature of raw satellite products. Optical satellite sensors do not acquire RGB images but a wider variety of spectral bands, which may have different spatial resolution. We propose a NeRF framework to simultaneously handle panchromatic data and lower resolution spectral bands (e.g., color bands), and investigate the contribution of the low-resolution bands to the output model. Our method achieves comparable or better results with respect to previous approaches that rely on a separate pansharpening step. The model can also be used to generate a pansharpened image surrogate for each input view, as it natively performs super-resolution in the color bands.