An Efficient Human-Computer Interaction in Battlefield Environment via Multi-stream Learning

Human-computer interaction is fundamental for the increasingly complicating battlefield intelligent and automotive. This task, however, is challenging owing to the vibration and uneven illumination environment of weapon equipment. The traditional interaction methods cannot meet the needs of fast-paced, high flexibility and long-distance operation. In this paper, we propose a multi-stream learning algorithm that uses gesture recognition to realize human-computer interaction. Our method utilizes Time Domain Aggregation and Domain Adaption Fusion modules to solve the error-prone identification problem under uneven illumination and vibration environment respectively. Experiments using 10 types of gestures and more than 3000 infrared and visible light images under the real mobile vehicle environment datasets demonstrate the robustness, accuracy and efficiency of our method compared to previous state-of-the-art gesture recognition methods.