Development and Evaluation of a Webcam-Based Gaze Control System for Mobile Manipulator

Mobile Manipulators possess significant potential to be utilized in the service industry, specifically in the form of assistive robots. This is evidenced by its hybrid composition and various applications in the agricultural, military, and industrial sectors. Traditional remote-control operations for mobile manipulators would not be suitable for individuals with motor disabilities, while EMG-based and EEG-based control systems possesses challenges in intuitive operation and data sensitivity which can lead to inaccuracies in the operation. With the challenge of determining the appropriate control system for assistive robots, this study develops a gaze-control system that utilizes a common webcam catered to individuals possessing motor disabilities as it is relatively accurate, intuitive, and fast. The developed control system employs the concept of Histogram of Oriented Gradients, Support Vector Machines, and Convolutional Neural Network for gaze detection and classification. The experimentations are divided into two parts, the evaluation of the control system and the performance of the control system for mobile manipulator operation. Results from the former experiments showed that the system has an overall average accuracy of 93.86% for calibrated users and 84.76% for uncalibrated users. The latter experiment showed that task conducted was successful nine out of ten iterations.

I. Introduction

Mobile manipulators are considered as hybrid robotic systems consisting of a mobile base that is responsible for its locomotion, and a manipulator which is responsible for its interaction with environment [1–3]. With the versatility of mobile manipulators, research studies have explored and verified its applicability in the field of agriculture by evaluating its capacity in seed sowing, fertilizer spraying, and harvesting [4–8]. Other mobile manipulator applications can be found in military [9–11], industrial [12–13], and service [14–15] industries where pick and place operations and environment interactions are necessary.