A Standardized Benchmark for Humanoid Whole-Body Manipulation

In this paper we focus on the evaluation of humanoid manipulation skills while balancing on two feet. This involves manipulation while standing and loco-manipulation where the object is being manipulated while taking steps. With this objective in mind, an initial study of whole-body manipulation in a box manipulation scenario with two dif-ferent motions using the University of Waterloo's REEM-C, “Seven”, is investigated to provide insight into a valuable setup, comprehensive test protocols and useful performance metrics based on real world data. The contribution of this paper is a proposed benchmark for whole-body manipulation consisting of the design of a test bed inspired by real use cases for humanoid whole-body manipulation tasks, the definition of a set of protocols to standardize the testing procedure and insightful key performance indicators (KPIs) based on this initial study with the real robot. The proposed benchmark for humanoid whole-body manipulation is part of the EUROBENCH project that aims at creating a benchmarking framework for robotic systems performing locomotion related tasks.