A Framework for an Intelligent RoboticManipulator Coupled with a Wheelchair

This paper proposes a framework for a robotic arm coupled with a wheelchair, to aid amputees in accessing physical objects. The framework comprises primarily the software modules with a hardware prototype that provides an environment to test the functionality. The hardware makes use of a stereo vision camera to provide a video feed of the physical objects present in front of the wheelchair, in addition to obtaining a series of three-dimensional point clouds corresponding to the video feed. The hardware includes a provision for two revolute joints offering pitch and yaw motion, located on a moving chair. The system actuates the motors to control the rotation angles towards picking up the objects. A Convolution Neural Network (CNN) model is used in the vision system for locating and segmenting the physical objects on each frame of the video feed, leading to the drawing of rectangles bounding the detected objects on the frames and displaying them on a touchscreen interface. When the user chooses an object via touch, the coordinates of the object are obtained via the three-dimensional point cloud, which is converted to its corresponding azimuth and elevation angles for actuation. The result formulates the mathematical equations for calculating azimuth and elevation with which the software is realized to establish control of locations for picking up the desired objects. The software running on the hardware prototype acts as a demonstrator to provide an alignment of the actuators with the chosen physical objects.