A Navigation-Based Evaluation Metric for Probabilistic Occupancy Grids: Pathfinding Cost Mean Squared Error

While robotics increasingly relies on occupancy grids for environment perception, the lack of specifically-designed metrics leads existing research to employ Image Quality Assessment (IQA) metrics and topological evaluations, which were primarily designed for binary occupancy grids. While appropriate as a first approximation, not taking into account the particular nature and usage of probabilistic occupancy grids limits the accuracy of their evaluation. In this paper, we propose the PathFinding Cost Mean Squared Error (PFC-MSE), a new probabilistic occupancy grid comparison metric designed to incorporate their main usage and attributes. Emulating grid-based navigation methods, the metric defines the difference between two grids as the measured spread between the navigation behavior their use induces, which emphasizes variations in general topology over local cell-value fluctuations. Experimental results on 10,000 driving scenes exhibit the relevance of the approach in quantifying grid disparities compared to existing approaches.