Multi-Objective End-to-End Self-Driving Based on Pareto-Optimal Actor-Critic Approach

The end-to-end control method is one of the ways to realize autonomous driving. Existing end-to-end self-driving approaches commonly just consider one objective, for instance, safety or fuel efficiency. To optimize multiple objectives simultaneously, Pareto-optimal driving policies need to be obtained to satisfy different driving requirements. However, it is difficult to obtain the Pareto-optimal driving policy set because of the high-dimensional and complicated input in the end-to-end self-driving task. To handle this problem, we propose the Pareto-optimal actor-critic approach, in which the policy updating rule considers whether the gradients of value functions corresponding to different objectives with respect to the action are the same. The proposed algorithm is independent with objective preference, unsusceptible to the character of the concavity and convexity of the Pareto front, and effective when dealing with the end-to-end self-driving problem. The proposed algorithm is evaluated on TORCS, and it outperforms the classical and state-of-the-art multi-objective reinforcement learning approaches.