Real-Time Full-Body Motion Capture from Video and IMUs

A real-time full-body motion capture system is presented which uses input from a sparse set of inertial measurement units (IMUs) along with images from two or more standard video cameras and requires no optical markers or specialized infra-red cameras. A real-time optimization-based framework is proposed which incorporates constraints from the IMUs, cameras and a prior pose model. The combination of video and IMU data allows the full 6-DOF motion to be recovered including axial rotation of limbs and drift-free global position. The approach was tested using both indoor and outdoor captured data. The results demonstrate the effectiveness of the approach for tracking a wide range of human motion in real time in unconstrained indoor/outdoor scenes.

1. Introduction

Real-time capture of human motion is of considerable interest in various domains including entertainment and the life sciences. Recent advances in computer vision and the availability of commodity wireless inertial sensors [13], [3] are beginning to take motion capture from constrained studio settings to more natural, outdoor environments, and with less encumbrance of the performer from specialized costumes and optical marker setups traditionally required (e.g. [4], [2]), while still retaining a high level of capture fidelity.