An Experimental Setup for Robot Learning From Human Observation using Deep Neural Networks

Industrial robots have gained great popularity in the last twenty years and become an integral component in many sectors empowering automation. Typically, trained technicians program robots before being commissioned into a production environment to perform a specific job. However, this method is suitable when the working environment is well structured with fixed parameters. Learning from demonstration (LfD) has emerged as an alternative robot programming paradigm for working in a dynamic production atmosphere with several uncertainties to obviate re-programming robots when something changes in the workplace. In this paper, we developed an LfD system based on computer vision to observe human actions and deep learning to perceive the demonstrator’s actions and manipulated objects. Object detection, object classification, pose estimation, and action recognition tasks are performed by deep neural networks. The proposed LfD method was demonstrated and tested with industrial robots for a pick-and-place application with 95% accuracy.