Remote Control of Excavator with Hand Pose

The excavator is important and widely-used. However, the traditional operation method is non-intuitive, with a fixed seat and restricted visual angle, bringing potential danger [1]. To improve this, this paper proposes a device named 3D mouse’ and a hand pose-based operation method as well as the system implementation utilizing it. The 3D mouse’ makes controlling a bucket in a similar way to using a computer mouse, both of which use hand pose. To implement the system, this paper covers algorithms of hand pose estimation based on human arm kinematics model, and a workspace mapping strategy to find the corresponding target bucket pose based on two reference points. Besides, an interpolation method and a closed-loop control architecture are designed, and forward and inverse kinematics of the excavator are discussed. Finally, the efficiency of this system was evaluated through filed tests.