Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations

Neural implicit functions have emerged as a powerful representation for surfaces in 3D. Such a function can en-code a high quality surface with intricate details into the parameters of a deep neural network. However, optimizing for the parameters for accurate and robust reconstructions remains a challenge, especially when the input data is noisy or incomplete. In this work, we develop a hybrid neural surface representation that allows us to impose geometry-aware sampling and regularization, which significantly improves the fidelity of reconstructions. We propose to use iso-points as an explicit representation for a neural implicit function. These points are computed and updated on-the-fly during training to capture important geometric features and impose geometric constraints on the optimization. We demonstrate that our method can be adopted to improve state-of-the-art techniques for reconstructing neural implicit surfaces from multi-view images or point clouds. Quantitative and qualitative evaluations show that, compared with existing sampling and optimization methods, our approach allows faster convergence, better generalization, and accurate recovery of details and topology.

1. Introduction

Reconstructing surfaces from real-world observations is a long-standing task in computer vision. Recently, representing 3D geometry as a neural implicit function has received much attention [40], [36], [7]. Compared with other 3D shape representations, such as point clouds [29], [20], [54], polygons [24], [31], [39], [32], and voxels [8], [47], [35], [22], it provides a versatile representation with infinite resolution and unrestricted topology.