A Real-Time Assistance System with Virtual-Real Fusion for Similar Components Assembly

Assembly tasks are common in industrial production and daily life, however, paper instructions usually lack expressiveness and interactivity. In addition, components often have similarities in appearance and shape, which are not considered by traditional pose tracking methods and thus perform poorly. This paper proposes a real-time assembly assistance system, especially for similar parts assembly scenarios to improve working efficiency and user experience. The system obtains 6-DoF pose of each part for 3D synchronous display, detects assembly state, corrects wrong steps, and provides guidance for the next step. To cope with the problem of multiple similar parts obscuring each other during the process, our method VOSCO utilizes overall and positional consistency constraints to effectively segment target objects for the subsequent pose estimation FICP. Additionally, in the accelerated point cloud alignment operation, we compute the depth gradient by an optimized discrete differentiation operator and use it as the basis for point cloud sampling, which dramatically improves computational speed while maintaining alignment accuracy. Both VOSCO and FICP achieved good results in the custom dataset BP-SEG and BP-POSE for similar parts assembly scenarios, and the operation of the system satisfies real-time and good user experience.