Self-Supervised Representation Learning for Motion Control of Autonomous Vehicles

To address the generalization problem of imitation learning-based end-to-end driving methods in unseen environments, we propose a self-supervised representation learning framework for motion control of autonomous vehicles (SRLMC). SRLMC first efficiently learns spatiotemporal representations invariant across different driving views using attentive Siamese 3D-CNNs and contrastive loss and thereby improving the generalization capability of multi-task leaning networks for vehicle motion control. SRLMC is trained on a large-scale driving dataset and evaluated with discrete and continuous control command prediction tasks. The experimental results and comparison to prior works confirm that the proposed SRLMC is effective and generalizable to unseen environments.