Center-of-Mass-Based Object Regrasping: A Reinforcement Learning Approach and the Effects of Perception Modality

The adaptive grasp of unknown objects has been a long-term challenge in robot manipulation. It has been shown that the center of mass (CoM) plays an important role in human natural grasp. In this article, a CoM-based grasp framework with a reinforcement learning (RL) approach is proposed, where the RL agent finds the optimal grasp pose and object's CoM by using the data from vision, finger tactile sensor and wrist force-torque sensor. We generate objects with different dynamic parameters (mass, CoM position, and friction coefficient) in the PyBullet simulation environment, then stochastic and deterministic policy are, respectively, used for regrasp training. The effects of multimodal fusion perception and unimodal perception on regrasp efficiency are compared in both RL policies. The learned CoM grasping agent is transferred to real robotic hardware and evaluated on 28 types of unseen household objects, including specifically 5 objects with dynamically changing CoM. A 98.8% success rate with an average of 2.15 times of regrasp is achieved. We also demonstrate that our method improves the regrasp efficiency compared with the baseline methods.