A Demonstration-Based Instruction System for Handling Manipulation of Deformable Objects With Overlaying Hand Postures and Contact Forces

An effective human resource development system is desirable for mass-production sites, such as sewing factories, which rely on skilled human resources. The manipulation of objects such as clothes, cushions, and stuffed toys during production can deform them, making it difficult to either transfer them directly among people or introduce automation technology to do it. In this study, considering this, we developed a demonstration-based teaching system. The proposed system presents a learner with an instructional video that overlays not only the posture of the skilled operator’s hand, but also the contact force. This allows the learner to know the manipulated part of the object, the employed hand posture, and the applied force, even if the occlusion problem occurs. The proposed system does not require learners to wear any auxiliary devices for receiving instructions while learning, making it highly convenient. We implemented the system and confirmed that it could successfully provide instructions for handling the issue of deformable object manipulation.