Interactive Multi-Objective Reinforcement Learning for Continuous Robot Control

Many real-world continuous control problems inherently involve multiple conflicting objectives, especially in the case of robotic control problems. In recent years, with the development of deep multi-objective reinforcement learning (MORL), these problems have been preliminarily solved in many scenarios. However, the role of the decision-maker (DM) is often overlooked. The existing methods often find many tradeoff policies, from which the DM needs to manually select the ones they truly need, which is often very difficult in real-world problems. In this paper, we propose an interactive MORL algorithm called I-MORL. It includes a rank model, which learns the latent preferences of the DM through a small amount of interaction. Then, this rank model, together with a proposed prediction model, guides the algorithm to accurately find the non-dominated policies that align with the DM's preferences. We compare I-MORL with baseline algorithms on five benchmark robot control problems. The results demonstrate that our method outperforms the baseline algorithms.