Robot Task Primitive Segmentation from Demonstrations Using Only Built-in Kinematic State and Force-Torque Sensor Data

This paper presents a method for segmentation and classification of kinesthetic demonstrations of robot peg-in-hole tasks using a deep neural network. The presented method depends only on sensor data that is readily available on collaborative robots (kinematic state and forcetorque readings) and does not need any additional sensors. Our method can be used to automatically derive program structures from a single demonstration of a robot task. This can reduce programming time, and make it easier to revise sections of a larger task. We introduce a combined architecture consisting of a Convolutional Neural Network block for raw feature extraction and a Long Short-Term Memory block for tracking the time-evolution of these features. We also extend the model from binary insertion segmentation to multi-class segmentation including pick up, place, free air motions, insertions, and extraction. Through an ablation study and comparison to previous work, we show that the new model performs better on the test set, containing one demonstrator, but significantly better on a generalization set consisting of ten previously unseen demonstrators, reaching an overall accuracy of 70%, which is 15 percentage points better than the closest-performing other method.