BLOCSIE - Benchmark for LOCalization in a Simulated Industrial Environment

Benchmarking of localization algorithms for autonomous mobile robots has become increasingly important as numerous algorithms have already been developed and continue to be proposed. To facilitate an objective comparison between various localization techniques based on different sensor types having specific sensor characteristics, e.g., LIDAR or color-range cameras (RGB-D), we propose the BLOCSIE (Benchmark for LOCalisation in a Simulated Industrial Environment) evaluation suite as a possible solution. It provides sensor simulation using realistic noise models, different scenarios, and the ability to use either predefined or self-defined paths for the creation of a dataset. Furthermore, it offers a toolkit for evaluating algorithm outputs, establishing a full end-to-end pipeline for benchmarking of mobile robot localization algorithms. In this paper, we provide a detailed description of BLOCSIE's capabilities. Based on an exemplary application, we highlight its functionalities as well as the importance of using a realistic noise model when analyzing the performance of localization algorithms.