Early Fusion Based CNN Architecture for Visual Servoing Systems

Visual feedback control refers to the usage of image data to design the motion of a robotic system. This type of problem is equivalent to generating linear and angular velocities that will drive a robotic systems from an initial image to target one. Classic visual servoing methods have disadvantages, such as high challenges on extracting and tracking visual features, regardless of the environmental conditions, and nonlinear dependencies regarding camera calibration. During the last years, these limitations have been alleviated by employing Convolutional Neural Networks (CNNs). The main goal of this work is to increase the performance of CNNs in visual feedback control expanding the neural input arrays with extra available data. For this, extra maps created via region-based segmentation are considered as input in an early fusion based architecture. These ready-to-use simplified descriptions of the initial and final layouts can help CNN understand the scenes, and compute accurate velocities. The role of the segmented maps is experimentally investigated on two different architectures that exemplify the suggested design idea. The results show that CNNs with input fusion offer a better approximation of the linear and angular velocities, and proper robustness to segmentation errors, as well.