Training Deep Neural Networks for Visual Servoing

We present a deep neural network-based method to perform high-precision, robust and real-time 6 DOF positioning tasks by visual servoing. A convolutional neural network is fine-tuned to estimate the relative pose between the current and desired images and a pose-based visual servoing control law is considered to reach the desired pose. The paper describes how to efficiently and automatically create a dataset used to train the network. We show that this enables the robust handling of various perturbations (occlusions and lighting variations). We then propose the training of a scene-agnostic network by feeding in both the desired and current images into a deep network. The method is validated on a 6 DOF robot.

I. Introduction

The goal of visual servoing (VS) techniques is to control a dynamic system, such as a robot, by using the data provided by one or multiple cameras [13], [6]. Classical approaches rely on the extraction, tracking and matching of a set of visual features. These features, generally points, lines, or moments, are used as inputs to a control law that allows a robust positioning of the robot. While there has been progress in extracting and tracking the relevant features, a new approach called direct visual servoing (DVS, eg, [10], [7], [16]) was introduced recently to consider the image as a whole, which does not require anymore feature extraction nor tracking. The main drawback of DVS is its small convergence domain compared to classical techniques, which is due to the high non-linearities of the cost function to be minimized.