3D Semantic Segmentation Algorithm for Indoor Scenes based on Long-term Memory

Deep learning has a strong ability to tackle pixel- level labeling tasks in image understanding. However, the disorder and irregularity of 3D point cloud data make it difficult to be applied. Though there are a few approaches that have acquired satisfied results on 3D data processing recently, there is no consideration of the relationships between each point, which plays an important role in indoor objects semantic segmentation and has been proved on plane images. In this paper, a spatial long-term connection of points is modeled by a fully connected conditional random field (CRF) to enhance the memory ability of the network. Since it is infeasible to compute due to the dimension explosion caused by potentially fully connected CRF, a highly efficient method in utilizing linear combination of multiple Gaussian kernels is proposed to approximate it. Finally, the CRF has been integrated into the deep learning framework, making it trainable end-to-end. The test evaluation of the experiments show that this network reaches current benchmarks and have a strong performance on public 3D indoor datasets. From these results, it is concluded that the relationships of each pair of points are conductive to semantic segmentation for indoor scenes.