Seq2Act: A Sequence-to-Action Framework for Novel Shapes in Robotic Peg-in-Hole Assembly

Robotic peg-in-hole assembly is a critical task in manufacturing that often faces misalignment issues due to sensor inaccuracies and control mechanisms. Traditional methods for addressing these issues, while effective, have limitations in flexibility and efficiency. This paper presents Seq2Act, a new framework for generating sequential peg-in-hole data to address the challenges mentioned above, using physical simulation and imitation learning. The framework is powered by a transformer model that predicts the next action for insertion. By leveraging simulation, the framework generates data and strategies for peg-in-hole insertion across a wide range of shapes, from quadrilaterals to decagons. The model anticipates peg movements for alignment and insertion, achieving impressive success rates of 87.7% and 81.4% for seen and unseen shapes, respectively. This highlights its adaptability without heavy reliance on reinforcement learning or human-guided demonstrations. Our approach represents a significant advancement in robotic assembly, providing a solution that is both data-driven and adaptable to a variety of shapes and sizes.