A Digital Twin for Teleoperation of Vehicles in Urban Environments

Teleoperated driving (ToD) is increasingly considered as a fallback solution for autonomous driving. Up to now, ToD requires a highly reliable mobile network capable of transmitting multiple video streams with low latency. Recently, significant advancements have been made in vehicular sensors and perception algorithms, which have huge implications for the challenges faced in ToD. We envisage that a real-time digital twin that tracks the remote vehicle's environment will play a crucial role in reducing the required communication band-width and providing a more convenient teleoperator interface, ultimately enhancing safety of ToD in crowded environments. Furthermore, it would allow various degrees of cooperation between automated driving functionalities and human teleoperators. In this paper, the concept of digital twin for ToD is outlined and a proof of concept is implemented using a real-world vehicle simulator and a teleoperator hardware setup. A significant reduction in required bandwidth is reported by transmitting less video data and reconstructing the scene from the digital twin.