Point Clouds With Color: A Simple Open Library for Matching RGB and Depth Pixels from an Uncalibrated Stereo Pair

Current day robots often rely - for visual perception - on the coupling of two cameras: one for color and one for depth. While for custom RGB-D cameras, the manufacturer takes care of aligning the two images, this is not done when two commercial cameras are coupled (e.g., on the Pepper robot) without having been calibrated beforehand. In this article, we present a simple open library for reconstructing the 3D position of RGB pixels without knowing the parameters of the two cameras. The library requires a simple preliminary calibration step based on pixel-to-pixel matching, and then automatically reconstructs 3D colored point clouds from a given set of pixels in the RGB image. The source code is available at the following link https://github.com/jordan-nowak/OpenHSML.