A mixed-reality framework based on depth camera for safety testing of autonomous navigation systems

Simulation testing is an essential stage to preparing vehicles for a variety of possible and dangerous situ-ations in order to validate autonomous driving algorithms on mobile systems. However, transferring the model issued from the algorithms to reality can be challenging. Mixed-reality environments facilitate testing models on real vehicles with reduced financial and safety risks. By introducing virtual elements to the agent's environment perception, mixed-reality frameworks can reduce the risks and costs associated with testing on real roads while enabling researchers to explore a greater number of potentially critical situations. This paper presents a mixed reality framework based on depth cameras. The framework uses an augmentation approach to combine objects from two environments (virtual and real) in a single world. The implementation details of the proposed method are discussed, and the qualitative and quantitative analysis of the experimental results demonstrate the potential of the proposed framework.