Interaction-Based Active Perception Method and Vibration-Audio-Visual Information Fusion for Asteroid Surface Material Identification

Asteroid exploration is challenging due to its unknown surface material and microgravity. This article proposes an interaction-based active perception (IBAP) framework and a vibration-audio-visual information fusion (VAVIF) method for asteroid surface material identification. First, a robotic sensor node was designed to acquire vibration and audio signals when impacting the surface. With the help of the “interaction” between the node and surface material, a camera records visual images of the material splashing. Then, we proposed a SimpleCNN (SCNN) model to explore the rich features of the fusion images converted from the vibration signals for material identification. We also proposed a bidirectional RNN with attention mechanism (AB-RNN) model to fuse the high- and low-frequency vibration signals to improve the recognition performance. Results showed that SCNN and AB-RNN have better performance than the state-of-the-art models. In addition, state-of-the-art machine learning models are used to classify the visual images and audio signals for identification. Moreover, we utilized the output of multiple classifiers to construct a new dataset and employed ensemble learning (EL) for training and testing. The EL-based VAVIF obtained a recognition accuracy of 99.7%, higher than any individual learner. The results indicate that IBAP and VAVIF make material identification more accurate and robust, which can improve the success rate of an asteroid exploration mission.