Genetic Imitation Learning by Reward Extrapolation

Imitation learning demonstrates remarkable performance in various domains. However, imitation learning is also constrained by many prerequisites. The research community has done intensive research to alleviate these constraints, such as introducing a stochastic policy to avoid unseen states, eliminating the need for action labels, and learning from the suboptimal demonstrations. Inspired by the natural reproduction process, we proposed a method called GenIL that integrates genetic operations with imitation learning. The involvement of the genetic operations improves the data efficiency by reproducing trajectories with various returns and assists the model in estimating more accurate and compact reward function parameters. We tested GenIL in both Atari and Mujoco domains, and the empirical result shows that it successfully outperforms the previous extrapolation methods in terms of extrapolation accuracy and overall policy performance when input data is limited. Code, data, and supplementary materials are publicly available at https://github.com/BryanZ666/GenIL.