Cross-overlapping Hierarchical Reinforcement Learning in Humanoid Robots

In the RoboCup3D project, how to make the humanoid robot with faster running speed and more accurate kicking action is a popular research direction. In this paper, we extend the Overlapping Layered Learning method by proposing a cross-overlapping hierarchical reinforcement learning method, which is based on overlapping layered learning to smooth the action articulation by cross-learning the articulated action parameters or cross-learning the higher-level action parameters to obtain better action execution. The article also introduces the baseline-based optimization technique and elaborates the specific optimization strategy and optimization task. Finally, the effectiveness of cross-overlapping hierarchical reinforcement learning and baseline-based optimization techniques is demonstrated experimentally.