Robust Multiview 3D Pose Estimation Using Time of Flight Cameras

In this work, we introduce a robust multi-view 3D pose estimation method which leverages infra-red intensity image and the depth information from the ToF camera. We use the intensity image as an input to a CNN based 2D key point detector from multiple view ports and forward project them to calculate the 3D joint position. If a keypoint is missing in one of the views we benefit from the depth information from the ToF camera. This results in a robust system which is robust against material reflectivity and occlusion. Our experiments show that the system is able to deliver 3D joint positions with a maximum positional error of only 4 cm independent of the material reflectivity. We also compare our technique with different 3D key point estimation techniques. We compare our system to a state-of-the-art CNN based multi-view depth based pose estimation technique to lift 2D key points from a single image to 3D pose. The maximum euclidean distance from the ground truth and its deviation for each system are evaluated against VICON, a marker based motion capture system.