Impact Sound-Based Surface Identification Using Smart Audio Sensors With Deep Neural Networks

Humans have the extraordinary ability to identify objects by listening to the sound that is produced when hitting the objects' surface. In this study, we propose an integrated sensor-actuator system that has the capability to identify the surface on which it is placed. The system contacts (i.e., taps or hits) the surface by generating a mechanical impact and then analyzes the resulting sound waveform to decipher the intrinsic characteristics of the surface on which it is placed. Further, a machine learning pipeline based on recent deep learning techniques was introduced to identify ten different everyday surfaces. The results demonstrated that the proposed system can successfully recognize various surfaces with a test accuracy of 91.73%. Furthermore, it was found that the proposed machine learning pipeline can be implemented in an embedded machine with an inference time of less than 300 ms. We conclude this paper with limitations and discussion of future work.