Learning Pre-Grasp Manipulation of Multiple Flat Target Objects in Clutter

Limited to the width of the robot's gripper, it is difficult to vertically capture flat objects such as books and plates, especially the flat target(s) may be one or more in clutter, and the task will be more challenging. The pre-grasp manipulation can make the object rearranged and move to the edge of table to realize it graspable. In this paper, the task is transformed into Parameterized Action Markov Decision Process, and the method is proposed to solve this problem based on deep reinforcement learning. In order to improve the data utilization, the weight sharing policy network is used to predict the sliding primitive parameters of each object, and then system selects the optimal execution object from all objects by the Q network. Considering the noise and complexity of the original image, we adopts the mask images of objects and the desktop as a kind of state of policy. Meanwhile, for enhancing action efficiency in the presence of multiple flat targets, an extra reward mechanism is added. In the simulation, our method realizes the pre-grasp manipulation with higher task success rate and fewer action steps.